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15.70  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  3,  by  level  of  parents'  education  373 

15.71  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  3,  by  articles  in  the  home  374 

15.72  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  3,  by  television  viewing  375 

15.73  Weighted  response  percentages  and  general  readmg 

proficiency  means,  grade  7,  total  sample  376 

15.74  Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  7,  by  sex  of  subject  377 

15.75  Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  7,  by  regjon  of  country  378 

15.76  Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  7,  by  derived  race  379 

15.77  Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  7,  by  level  of  parents'  education  380 

15.78  Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  7,  by  articles  in  the  home  381 

15.79  Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  7,  by  television  viewing  382 

15.80  Weighted  response  percentages  and  general 

mathematics  proficiency  means,  grade  7,  total  sample  383 

15.81  Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  7,  by  sex  of  subject  384 

15.82  Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  7,  by  region  of  country  335 

15.83  Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  7,  by  derived  race  335 

15.84  Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  7,  by  level  of  parents'  education  387 

15.85  Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  7,  by  articles  in  the  home  388 

15.86  Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  7,  by  television  viewing  389 

15.87  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  total  sample  39O 

15.88  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  by  sex  of  subject  391 

15.89  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  by  region  of  country  392 

15.90  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  by  derived  race  393 

15.91  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  by  level  of  parents'  education  394 

15.92  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  by  articles  in  the  home  395 

15.93  Weighted  response  percentages  and  general  science 

proficiency  means,  grade  7,  by  television  vievvdng  395 
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Table  15.94 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  total  sample 

397 

15.95 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  by  sex  of  subject 

398 

15.96 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  by  region  of  country 

399 

15.97 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  by  derived  race 

400 

15.98 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  by  level  of  parents'  education 

401 

15.99 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  by  articles  in  the  home 

402 

15.100 

Weighted  response  percentages  and  general  reading 

proficiency  means,  grade  11,  by  television  viewing 

403 

15.101 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  11,  total  sample 

404 

15.102 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  11,  by  sex  of  subject 

405 

15.103 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  11,  by  region  of  country 

406 

15.104 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  li,  by  derived  race 

407 

15.105 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  11,  by  level  of  parents'  education 

408 

15.106 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  11,  by  articles  in  the  home 

409 

15.107 

Weighted  response  percentages  and  general  mathematics 

proficiency  means,  grade  11,  by  television  viewing 

410 

15.108 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  total  sample 

411 

15.109 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  by  sex  of  subject 

412 

15.110 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  by  region  of  cour  ly 

413 

15.111 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  by  derived  race 

414 

15.112 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  by  level  of  parents'  education 

415 

15.113 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  by  articles  in  the  home 

416 

15.114 

Weighted  response  percentages  and  general  science 

proficiency  means,  grade  11,  by  television  viewing 

417 

12 
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EXPANDING  THE  NEW  DESIGN: 
THE  NAEP  1985-86  TECHNICAL  REPORT 


INTRODUCTION 

Albert  E.  Beaton 
Educational  Testing  Service 


The  1986    National  Assessment  of  Educational  Progress  (NAEP)  surveyed 
what  students  knew  and  could  do  in  the  subject  areas  of  reading,  mathematics 
science,  computer  competence,  and,  for  older  students  only,  U.S.  history  and' 
literature.     The  populations  that  were  sampled  included  American  public  and 
private  school  students  of  ages  9,  13,  and  17  as  well  as  those  in  3rd  7th 
and  11th  grades.  ' 

This  technical  report  is  intended  to  supply  the  details  of  the  design 
and  data  analysis  of  the  1986  assessment.     Our  aim  is  to  give  the  reader 
sufficient  information  to  judge  the  utility  of  the  design,   the  quality  of  the 
NAEP  data,  the  reasonableness  of  the  assumptions  made,  the  appropriateness  of 
the  data  analyses,  and  the  generalizability  of  the  inferences  made  from  the 
data.     This  report  covers  only  the  technical  aspects  of  the  1986  NAEP.  It 
does  not  attempt  to  provide  the  substantive  results  that  might  be  of  interest 
to  educational  policymakers;  such  results  are  provided  in  the  NAEP  cross - 
sectional  and  trend  reports  on  student  achievement  in  the  various  subject 
areas.     This  technical  documentation  is  intended  to  support  the  proficiency 
reports  by  presenting  detailed  iiiformation  about  the  data  and  analyses  that 
were  interpreted  and  presented  in  tht.  reports.     Analyses  performed 
specifically  for  the  achievement  reports  are  discussed  in  the  procedural 
appendices  of  those  reports. 

The  NAEP  staff,  of  course,  did  not  do  this  work  alone.     It  had  the 
policy  guidance  of  the  Assessment  Policy  Committee  (APC) ,  chaired  by  Senator 
Pat  Frank  of  Florida.     It  is  also  important  to  recognize  the  many  thoughtful 
discussions,  recommendations,  reviews,  comments,  and  other  substantial  help 
on  technical  issues  that  the  NAEP  staff  received  from  the  highly  accomplished 
members  of  its  Design  and  Analysis  Committee  (formerly  the  Technical  Advisory 
Committee)  chaired  by  Professor  Robert  Linn  (University  of  Colorado)  Other 
members  of  this  committee  are  Professor  John  B.  Carroll  (University  of  North 
Carolina),  Professor  Robert  Glaser  (University  of  Pittsburgh),  Professor  Bert 
Green  (Johns  Hopkins  University),  Professor  Sylvia  Johnson  (Howard 
University),  Professor  Ingram  Olkin  (Stanford  University) ,  Dr.  Tej  Pandey 
(California  Department  of  Education) ,  Professor  Richard  Snow  (Stanford 
University),  and  Professor  John  W.  Tukey  (Princeton  University). 

^Throughout  this  report,  the  assessments  conducted  during  the  1983-84 
and  1985-86  school  years  will  be  referred  to  respectively  as  the  1984  and 
198o  assessments. 


This  technical  report  is  organized  into  three  parts: 

Part  I  begins  with  Chapter  1,  which  contains  a  discussion  of  the  NAEP 
1986  design  and  a  summary  of  the  steps  followed  in  the  process  of  developing 
the  basic  data.     This  chapter  is  followed  by  chapters  covering  the 
development  of  the  items  in  the  subject  areas  that  were  assessed;  the 
sampling;  the  measurement  instruments;  the  field  administration  (including 
attainment  of  school  cooperation);  and  the  data  entry,  item  scoring,  and 
construction  of  the  NAEP  database  and  public-use  data  tapes.     Quality  control 
is  covered  throughout  Part  I . 

Part  II  explains  the  steps  involved  5.n  data  analysis.     Chapter  7,  the 
first  chapter  in  this  part,  begins  with  an  overview  of  the  aims  of  the  data 
analysis  and  a  summary  of  the  procedures.     Chapter  8  contains  an  overview  of 
the  scaling  procedures  used  in  NAEP  and  recommended  analytic  procedures. 
Subsequent  chapters  describe  the  application  of  the  data  analytic  procedures 
in  reading,  mathematics,  science,  computer  competence,  and  history  and 
literature.     The  final  chapter  in  Part  II  discusses  the  sampling  weights  and 
variance  estimation. 

Part  III  presents  some  basic  data  from  the  1986  NAEP,   including  the 
properties  of  the  measuring  instruments,  characteristics  of  the  selected 
sample,  and  some  estimates  of  the  proficiency  of  the  students  in  American 
schools.     Only  a  few  of  the  huge  number  of  possible  population  proficiency 
estimates  are  presented,  and  these  include  estimates  of  the  means,  standard 
deviations,  and  selected  percentiles  of  populations  of  students  in  various 
subject  areas  at  various  grade  levels.     Population  estimates  are  also 
reported  separately  by  gender,  racial/ethnic  grouping,  and  other 
subpopulations .     Estimates  of  the  average  values  for  cross -classifications  of 
several  important  variables  are  also  reported.     Estimated  standard  errors  are 
provided  with  all  parameter  estimates. 

The  rest  of  this  introduction  contains  general  background  information 
about  NAEP  and  comment  on  the  rationale  for  some  of  the  modifications  that 
have  been  implemented  in  design  and  analysis.     The  section  gives  the 
legislative  mandate  under  which  this  assessment  was  conducted.  Subsequent 
sections  briefly  describe  the  changes  in  the  design  of  NAEP  in  the  1986 
assessment,  the  steps  in  implementing  NAEP,   important  changes  in  analytic 
procedures,  the  effect  of  the  several  changes  on  the  National  Assessment,  the 
rationale  for  the  public-use  data  tapes,  and  the  organization  strategy  of 
this  report. 

CONGRESSIONAL  MANDATE  FOR  NAEP 

The  National  Assessment  of  Educational  Progress  is  a  continuing, 
congressionally  mandated,  national  survey  of  educational  achievement.  The 
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[NAEP]     .shall  have  as  a  primary  purpose  the  assessment  of 
performance  of  children  and  young  adults  in  the  basic  skills  of 
reading,  mathematics,  and  communication.     Such  a  National 
Assessment  shall... 

(A)  collect  and  report  at  least  once  every  five  years  data 
assessing  the  performance  of  students  at  various  age  or  grade 
levels  in  each  of  the  areas  of  reading,  writing,  and  mathematics; 

(B)  report  periodically  data  on  changes  in  knowledge  and  skills  of 
such  students  over  a  period  of  time; 

(C)  conduct  special  assessments  of  other  educational  areas,  as  the 
need  for  additional  information  arises;  and 

(D)  provide  technical  assistance  to  State  educational  agencies  and 
to  local  educational  agencies  on  the  use  of  the  National  Assessment 
objectives,  primarily  pertaining  to  the  basic  skills  of  reading, 
mathematics,  and  communication,  and  on  making  comparisons  of  such 
assessments  with  the  national  profile  and  change  data  developed  by 
the  National  Assessment." 

In  addition  to  fulfilling  this  Congressional  mandate,  NAEP  also  gathered 
ancillary  data  that  can  be  of  use  in  interpreting  the  basic  findings  about 
the  knowledge  and  skills  of  young  Americans.     It  is  the  first  ongoing  effort 
to  obtain  comprehensive  and  dependable  achievement  data  on  a  national  basis 
in  a  uniform,  scientific  manner.     NAEP  was  originally  designed  in  the  1960s 
and  collected  its  first  data  in  1969.     The  NAEP  grant  was  administered  by  the 
Education  Commission  of  the  States  (ECS)  until  1983  when  the  grant  was  moved 
to  Educational  Testing  Service  (ETS) .     Since  its  inception,  NAEP  has 
collected  information  not  only  on  reading,  writing,  and  mathematics,  as 
required  by  the  1978  law,  but  also  on  a  number  of  other  subject  areas  such  as 
science,  citizenship,  art,  and  music. 

In  1982,  ETS  proposed  a  new,  comprehensive  design  for  NAEP.     The  design 
was  described  extensively  in  The  Conduct  of  the  National  Assessment  of 
Educational  Progress,  a  Proposal  in  Response  to  RFP  PA- 82 -001^  submitted  by 
ETS  to  the  National  Institute  of  Education,  November  17,  1982.     An  overview 
of  the  design  was  published  in  the  report  A  New  Design  for  A  New  Era 
(Messick,  Beaton,  &  Lord,  1983).     Five  years  have  passed  since  ETS  received 
the  grant  to  implement  its  design  for  NAEP;  the  concepts  in  the  proposed 
design  have  now  been  put  into  practice,  the  students  have  been  assessed,  the 
resulting  data  have  been  analyzed,  and  reports  have  been  published,  including 
a  previous  technical  report.  Implementing  the  New  Design:  The  NAEP  1983-84 
Technical  Report  (Beaton,  1987a). 

It  should  be  noted  that  the  Congressional  mandate  for  NAEP  has  been 
modified  and  broadened  to  augment  the  design  of  the  1990  and  ensuing 
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assessments  by  introducing  sununary  proficiency  estimates  for  individual 
states.  The  design  of  the  1990  assessment  will  be  published  in  a  separate 
document . 

CHANGES  IN  THE  NAEP  DESIGN 

Although  the  original  design  for  NAEP  was  "brilliantly  responsive  to  the 
political  constrains  of  the  time"  (Messick,  Beaton,  &  Lord,  1983,  p.l),  the 
structure  of  NAEP  was  changed  in  significant  ways  in  the  1984  assessment  and 
evolved  even  further  in  1986.  The  design  of  the  1986  National  Assessment  is 
discussed  fully  in  Chapter  1.  The  rationale  for  the  major  design  changes  is 
discussed  here. 

In  measuring  student  progress  over  time,   it  is  essential  that  NAEP 
remain  stable  in  order  to  maintain  the  comparability  of  the  results  from  one 
assessment  year  to  another.     However,  as  the  concerns  of  educational 
policymakers  change  and  as  new  technologies  become  available,   it  is  important 
to  modify  NAEP  in  order  to  increase  its  relevance  and  maintain  its  technical 
excellence.     This  tension  between  stability  and  change  has  presented,  and 
continues  to  present,  a  challenge  to  the  technical  staff  of  NAEP. 

Since  receiving  the  grant  in  1983,  ETS  has  instituted  a  number  of 
modifications  in  the  way  that  NAEP  is  administered  and  its  data  analyzed. 
Balanced  incomplete  block  (BIB)  spiralling  and  IRT  scaling  were  introduced 
in  the  1984  assessment  of  reading  and  writing.     Another  modification  to  that 
assessment  was  the  additicii  of  grade-level  sampling.     Data  were  collected  not 
only  for  students  aged  9,   13,  and  17  (as  in  past  assessments)  but  also  for 
students  in  grades  4,  8,  and  11.     These  changes  have  been  reported  previously 
(Messick,  Beaton,  &  Lord,   1983;  Beaton,  1987a). 

In  1986,  reading,  mathematics,  science,  and  computer  competence  were 
assessed  at  grades  3,  7,  and  11  and  ages  9,   13,  and  17.     At  grade  11/age  17 
only,  U.S.  history  and  literature  were  also  assessed.     Since  NAEP  had  not 
assessed  computer  competence,  history,  or  literature  before,  there  were  no 
trend  results  to  which  new  results  could  be  compared  and  thus  no  opportunity 
to  study  the  effects  of  changes.     However,  reading,  mathematics,  and  science 
had  been  assessed  several  times  in  the  past;  thus,  it  was  necessary  to  see 
what  effects  the  changes  in  methodology  had  on  the  trend  data  for  these  three 
subject  areas. 

There  were  two  major  differences  in  the  design  of  the  1986  assessment 
from  past  assessments; 

The  definitions  of  the  age  populations  and  the  time  of 
year  in  which  tlie  assessment  was  administered  were 
changed. 

Mathematics  and  science  were  assesc?.d  for  the  first  time 
using  only  printed  instructions  and  items,  eliminating 
the  paced  aural  presentation  by  tape  recorders  used  in 
past  assessments. 


The  changes  in  age  and  time  of  testing  were  introduced  to  make  NAEP 
definitions  uniform.     In  pre-1986  NAEP,  the  definition  of  age  for  17-year- 
olds  was  not  consistent  with  the  definitions  for  9-  and  13-year-olds  with 
the  result  that,  although  all  9-year-olds  would  be  eligible  for  the  IS-year- 
oid  sample  in  four  years,  most  of  NAEP's  13-year-old  students  would  not  be 
eligible  for  the  17-year-old  sample  four  years  later.     Also,  pre-1986 
assessments  sampled  13-year^olds  in  the  fall,  9-year-olds  in  the  winter,  and 
17-year-olds  in  the  spring,  with  the  result  that  the  apparent  four-year 
difference  between  age  samples  was  actually  something  different.     In  1986 
NAEP  introduced  consistent  definitions  of  age  and  assessment  of  all  ages  at 
the  same  time  of  year. 

The  change  from  tape-recorded  to  printed  administration  was  necessitated 
by  the  introduction  of  BIB  spiralling.     Before  the  1984  assessment,  NAEP  was 
administered  using  matrix  sampling  and  all  students  at  an  assessment  session 
were  assigned  exactly  the  same  items.     It  was  possible,  and  deemed  desirable, 
to  administer  the  assessment  using  a  tape  recorder.     However,  the  tape 
recorder  had  a  negative  effect  on  one  of  NAEP's  aims- -to  develop  items  that 
can  be  used  in  other  settings.     Few  other  users  of  N\EP  items  chose  to  use 
tape-recorded  administration.     Also,  since  BIB  spiralling  assigned  different 
assessment  items  to  different  students  in  the  same  session,  administration  by 
a  single  tape  recorder  was  no  longer  feasible.^    For  these  and  other  reasons, 
the  traditional  tape-recorded  administration  method  was  replaced  by  a  paper-' 
and-pencil  administration.     In  1986,  the  change  from  tape-recorded  to  printed 
administration  was  new  only  for  mathematics  and  science;   reading  had  already 
been  assessed  using  printed  administration  in  1984. 

The  main  strategy  used  to  protect  the  stability  of  NAEP  as  changes  are 
introduced  into  the  NAEP  design  has  been  the  "bridge  study."    A  bridge  study 
entails  collecting  two  separate  but  comparable  sets  of  data.     In  order  to 
maintain  comparability  ^ith  past  assessments,  one  rational  sample  of  students 
is  assessed  using  the  same  methods  as  in  the  past,  duplicating  as  closely  as 
possible  past  administrative  procedures.     Another,  equivalent  sample  is  also 
assessed,  using  the  new  m.ethods  and  procedures.     The  data  from  the  two 
samples  can  then  be  compared  and  the  effect  of  the  changes,   if  any, 
estimated.     If  the  two  sets  of  data  are  equatable,  then  the  data  from  the 
past  assessments  can  be  adjusted  to  make  them  comparable  to  the  new,  or  vice 
versa.     If  the  data  from  the  two  samples  are  so  different  as  to  be  non- 
equatable,  the  trend  is  continued  using  only  the  students  assessed  with  the 
older  methodology.     Future  assessments  can  be  made  comparable  to  either 
depending  on  whether  or  not  the  value  of  the  changes  outweighs  the  value  of 
the  trend  data.    Two  bridge  studies  were  conducted  in  1986:     one  to  study  the 
effect  of  changing  the  definitions  of  age  and  the  time  of  year  in  which  the 
assessment  occurred  and  another  to  study  the  effect  of  changing  the  method  of 
administration. 


2 

BIB  spiralling  is  described  in  Messick,  Beaton,  and  Lord  (1983)  and 
Beaton  (1987a).  ' 
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IMPLEMENTATION  OF  THE  NAEP  DESIGN 


The  implementation  of  the  1986  NAEP  design  involved  a  large  number  of 
steps  which  are  described  briefly  in  the  next  chapter  and  then  in  some  detail 
in  Chapters  2  through  6.     First,  Learning  Area  Committees  were  formed  to  set 
the  objectives  for  the  several  subject  areas  that  were  assessed.  The 
reading,  mathematics,  and  science  items  from  past  assessments  were  reviewed 
and  reorganized,  and  new  items  developed  and  pretested  in  mathematics, 
science,  computer  competence,  U.S.  history,  and  literature.     Background  and 
attitude  questions  for  students  were  selected  or  developed.  Questionnaires 
for  teachers,  principals,  computer  coordinators,  and  excluded  students  were 
developed.     Altogether,   56  different  assessment  booklets  or  questionnaires 
were  printed  for  use  at  the  elementary  school  level,   72  at  the  intermediate 
level,  and  97  in  the  high  schools. 

Field  administration  was  performed  by  Westat.     Staff  were  recruited  and 
trained  in  the  randomly  selected  areas  of  the  country  where  the  assessment 
took  place.     Individual  states,  school  districts,  and  schools  were  encouraged 
to  participate.     The  assessment  was  administered  in  over  1,500  schools.  The 
various  forms  and  booklets  were  accounted  for  and  returned  to  ETS .  Quality 
control  procedures  were  implemented  and  reviewed. 

ETS  had  to  assure  that  it  received  complete  data  from  the  field 
administrators.     After  receiving  data,  ETS  professionally  scored  open-ended 
items  and  entered  the  data  into  its  computers.     While  most  of  the  data  were 
machine - readable ,  some  questionnaires  were  key-entered  and  verified. 
All  data  were  carefully  edited  and  placed  in  an  integrated  database.  Careful 
quality  control  checks  were  made  before  the  data  were  considered  ready  for 
data  analysis.     Westat  prepared  sampling  weights  foi.'  each  student  in  the 
sample.     ETS  then  placed  the  data  on  public-use  data  tapes  and  prepared  user 
docuiTentation . 

After  completion  of  the  database  with  sampling  weights,   the  data 
analysis  and  reporting  process  proceeded.     The  analysis  process  consisted  of 
reviewing  individual  items  to  assure  their  correctness,  then  developing 
anchored  scales.     The  scaled  data  were  analyzed  and  tabulated  in  many  ways. 
Reporting  interacts  with  analysis,  and  consists  of  identifying  the  issues  of 
interest,  specifying  data  analyses,  reviewing  and  interpreting  results,  and 
then  preparing  documents  for  review,  revision,  and  printing. 

CHANGES  IN  NAEP  ANALYSIS 

Part  II  of  this  report  describes  the  procedures  used  to  analyze  the  1986 
data.     These  procedures  are  summarized  in  Chapter  7  and  presented  in  detail 
in  Chapters  8  through  14.     In  this  section,  only  major  differences  in 
analytic  procedures  from  past  assessments  will  be  noted. 

Since  its  inception,  NAEP  has  had  as  its  goal  reporting  to  the 
interested  public  what  students  can  and  cannot  do.    NAEP  was  designed  to 
report  the  achievement  of  students  in  the  United  States  as  a  whole  and  in 
subpopulations  such  as  groups  based  on  regions  of  the  country,  ethnicity,  and 
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gerder  However,  the  way  in  which  these  goals  have  been  achieved  has  ev.^  .ved 
over  time. 

At  first,  NAEP  reported  what  students  knew  and  could  do  by  estimating 
and  publishing  the  proportion  of  students  who  could  pass  particular  items 
with  the  result  of  reporting  with  little  summarization  and  in  more  detail' 
than  most  audiences  wanted.     Reporting  procedures  were  quickly  modified  to 
focus  on  estimating  and  publishing  the  proportion  of  students  who  could 
answer  correctly  certain  specified  populations  of  items.     Although  the  data 
could  be  summarized  in  this  way.  the  approach  restricted  the  measurement  of 
change  over  time  to  groups  of  items  that  were  identical  in  several 
assessments  and  restricted  the  ways  in  which  the  data  could  be  interpreted. 

Since  receiving  the  NAEP  grant  in  1983.  ETS  has  used  scaling  technology 
to  report  student  achievement.  Using  scaling  technology  does  not  preclude 
alternative  reporting  to  maintain  continuity  with  the  past.     Although  NAEP 
now  emphasizes  scales  in  its  reports,   it  makes  its  data  available  on  fully 
documented  public-use  data  tapes  so  that  anyone  interested  may  find  out  how 
students  did  on  any  or  all  of  the  items  that  were  administered  For 
comparability  with  the  past.  NAEP  continues  to  publish  the  average  percenta^^e 
correct  for  groups  of  items  that  were  administered  at  different  times. 

The  scaling  within  NAEP's  different  learning  areas  is  heavily  dependent 
on  knowledge  of  the  interitem  correlations.  In  simplest  terms,   the  main  idea 
IS  that  If  the  items  could  be  placed  in  such  an  order  that  a  person's 
answering  an  item  correctly  at  a  particular  difficulty  level  implied  that  he 
or  she  could  answer  all  easier  items,  knowing  the  most  difficult  item  a 
student  could  answer  correctly  would  imply  what  that  student  could  and  could 
not  do  for  the  entire  population  of  items.     Of  course,  few.  if  any,  sets  of 
real  items  are  so  rigidly  ordered,  and  such  ordering  is  clearly  impossible 
where  guessing  is  allowed.     However,  other,  less  demanding,   item  response 
theory  (IRT)  models  are  available  to  be  applied  when  the  data  are 
approximately  unidimensional .     In  the  1984.  assessment  the  introduction  of 
BIB  spiralling,  a  complex  variant  of  multiple  matrix  sampling,  facilitated 
the  collection  of  interitem  data  in  such  a  way  that  dimensionality  could  be 
explored.  ■' 

_  If  the  dimensionality  study  showed  that  the  items  fell  approximately  on 
a  single  dimension,  a  single  scale  could  summarize  most  of  the  information 
about  student  performance  quite  adequately.     If  the  items  fell  on  more  than 
one  dimension,  a  scale  for  each  dimension  would  have  to  be  developed  if 
sufficient  data  were  available  to  support  the  scaling  process;  otherwise 
other  summarization  procedures,  such  as  the  average  percentages  used  in  the 
past,  could  be  employed.     The  1984  NAEP  showed  that  a  majority  of  the  reading 
Items  could  be  adequately  fit  to  a  unidimensional  model  and  so  these  reading 
Items  were  scaled.     Using  the  ordering  of  the  items,  the  reading  scale  was 
behayiorally  anchored  so  that  points  on  the  scale  could  be  interpieted  as 
levels  of  proficiency,  describing  what  students  at  those  levels  could  and 
T    .^'f  ^^^"^  writing  items  were  scaled  asing  an  alternative  method 

that  did  not  require  the  assumption  of  unidimensionality . 


In  1986,  che  scaling  technology  was  extended  to  subscales.     It  was 
assumed,  a  priori ,  that  a  single  scale  would  not  suffice  for  reporting 
mathematics  and  science.    Multivariable  scaling  technology  was  developed  to 
create  subscales  representing  various  categories  of  items  that  would  be  of 
interest  to  various  audiences.     Thus,  several  subscales  were  developed  for 
mathematics  and  for  science.     In  order  to  maintain  a  single  overall  measure 
of  performance  for  each  subject  area,  a  global  composite  was  also  created  for 
mathematics  and  for  science.     The  global  composite  was  anchored  for  each  of 
these  subject  areas  using  a  new  technology  developed  for  this  purpose. 


THE  EFFECT  OF  THE  CHANGES 

The  changes  in  the  design  and  analysis  of  the  National  Assessment  were 
intended  to  facilitate  analysis  and  reporting  of  NAEP  data,  and  have  done  so. 
Our  concern  here  is  whether  the  changes  in  design  have  maintained  the 
comparability  of  the  newly  collected  data  with  the  data  collected  by  NAEP 
since  its  first  data  collection  ir  1969-70, 

The  bridge  samples  that  were  collected  to  measure  the  effect  of  the 
changes  made  in  the  1986  NAEP  design  were  analyzed  and  the  results  are 
presented  in  a  separate  report  entitled  National  Assessment  of  Educational 
Progress  1986  Bridge  Studies  (Kaplan.  Beaton,  Johnson,  &  Johnson,  1988). 
That  report  concluded: 

"...the  effect  of  change  in  the  mode  of  administration,  while 
present,  is  accurately  predictable  and  does  not  differentially 
affect  the  measures  of  performance  for  any  of  the  subpopulations 
studied.     The  effect  of  the  change  in  age  definition  and  time  of 
testing  is  more  noticeable,  is  in  the  expected  direction,  and  does 
not  differentially  affect  the  measures  of  performance  for  the  vast 
majority  of  the  assessed  [sub]population[s] .     Consequently,  the 
data  from  the  past  assessments  can  justifiably  be  adjusted  to  make 
them  comparable  with  the  data  from  the  1986  assessment." 

However,  an  anomalous  situation  arose  in  the  analysis  of  the  reading 
data.     The  reading  data  were  collected  in  the  same  way  as  in  the  1984 
assessment,  to  which  performance  was  to  be  compared.     Analysis  of  these  data 
showed  an  inexplicable  decline  in  reading  proficiency,  a  decline  so  large  as 
to  be  suspect.     Eminent  outside  educational  researchers  also  examined  the 
NAEP  reading  results  and  concurred  with  the  NAEP  staff  in  their  judgment  that 
the  results  were  unusual  and  advised  further  investigation.     This  decline  was 
carefully  investigated  and  documented  in  The  NAEP  1985-86  Reading  Anomaly:  A 
Technical  Report  (Beaton,  1988).     That  report  states: 

"The  results  of  the  studies  of  the  reading  anomaly  are 
inconclusive.     Some  hypotheses,  such  as  inaccuracies  in  sampling, 
scaling,  and  quality  control,  can  be  ruled  out  beyond  any 
reasonable  doubt.     However,  some  changes  in  the  assessment  process 
are  inevitable,  and  these  changes  are  documented  in  this  report. 
The  possibility  that  one  or  a  combination  of  such  changes  may  have 


ao- 


resulted  in  the  declines  in  reading  proficiency  cannot  be  ruled 
out.     The  effect  of  such  changes  cannot  be  estimated  from  existing 
data."  ^ 

Since  the  reading  trend  data  have  been  deemed  anomalous,  the  reading 
trend  results  have  not  yet  been  published.     The  1988  NAEP  design  has  been 
modified  to  collect  additional  data  to  help  explain  or  sustain  the  unusual 
results  from  the  1986  reading  trend  analyses.     The  reading  trend  results  will 
be  published  after  the  new  data  are  collected  and  analyzed. 

THE  NAEP  DATABASE  AND  PUBLIC -USE  DATA  TAPES 

All  of  the  data  collected  during  the  1986  national  assessment  are 
available  on  public-use  data  tapes,  except  for  information  that  would  breach 
confidentiality  agreements  by  identifying  individual  states,  schools 
teachers  or  students.     These  tapes  are  fully  documented  in  the  National 
Assessment  of  Educational  Progress  1985-86  Puhlic-Use  Data  Tapes  Version  9  n 
Users'  Guide  (Rogers,  Kline,  Norris,  Johnson,  Mislevy,  Zwick,  Barone  & 
Kaplan,  1988), 

The  NAEP  staff  was  greatly  concerned  not  only  with  producing  its  own 
reports  but  also  with  making  its  public -use  data  tapes  available  in  a  format 
which  would  be  as  easy  for  others  to  use  as  possible.     The  purpose  of  the 
public-use  data  tapes  is  to  allow  others  to  check  our  analyses,   to  perform 
alternate  analyses  using  different  methods,  and  to  perform  analyses  for  other 
purposes.     The  public-use  data  tapes  are  formatted  for  and  have  parameter 
statements  for  thfe  commonly  used  statistical  systems  SAS  and  SPSS. 

Despite  the  substantial  effort  devoted  to  making  the  public -use  data 
tapes  as  easy  to  use  as  possible,   it  is  impossible  to  make  the  use  of  a 
database  as  complex  as  NAEP's  completely  simple.     A  secondary  user  cannot  use 
the  database  effectively  without  some  knowledge  of  the  NAEP  design  For 
example,  sampling  by  grade  and  age  forces  the  user  to  consider  which 
subsample  is  appropriate  for  a  particular  analysis.     BIB  spiralling  results 
in  a  substantial  amount  of  data  which  is  missing  by  design  (over  90 
percent!);  thus,  the  user  must  think  carefully  about  missing  data  procedures 
Although  we  have  tried  to  make  the  public-use  data  tapes  as  easy  to  use  as 
possible,  their  use  will  require  some  investment  in  understanding  NAEP. 

Two  features  of  the  tapes  give  the  user  additional  analytic  power.  Most 
complex  surveys  require  sampling  weights  to  achieve  proper  population 
estimates,  and  the  weights  are  supplied  for  use  in  analysis.     This  has  been 
done  for  NAEP.     However,  with  a  complex  sampling  design,   the  weighted 
versions  of  standard  formulas  for  independent  and  identically  distributed 
variables  are  not  appropriate  for  estimating  sampli..g  errors;  while 
appropriate  formulas  can  be  developed,   they  are  complex  to  apply.     Some  other 
method  based  on  pseudoreplicates ,  such  as  the  jackknife,   is  appropriate  and 
simple  m  application.     We  have  developed  and  applied  one  form  of  the 
jackknife  method,  which  we  have  used  in  all  NAEP  analyses.     It  requires  38 
sampling  weights  for  each  student  in' addition  to  the  sampling  weight  usually 


supplied.  All  of  these  weights  are  provided  on  the  public-use  data  tapes  in 
a  way  that  makes  possible  the  approximate  estimation  of  sampling  error  using 
standard  statistical  systems  as  opposed  to  specialized  software  designed  for 
survey  data.  Since  this  ability  comes  with  the  cost  of  more  computing  time, 
the  secondary  user  may  use  this  new  ability  or  not,  as  he  or  she  deems 
appropriate . 

The  other  feature  of  the  public-use  data  tapes  is  that  they  exceed  the 
standard  practice  of  providing  only  raw  data  by  also  providing  derived 
variables  for  reading  and  writing.     The  complexity  of  the  IRT  scaling 
analysis  prompted  this  inclusion.     The  underlying  rationale  follows. 

The  item-sampling  designs  that  have  characterized  NAEP  since  its 
inception  provided  efficient  estimates  for  average  levels  of  performance  in 
groups  of  students,  but  are  too  sparse  to  yield  accurate  estimates  for 
individual  students.     Until  1984,  NAEP  reported  only  estimates  of  the 
proportions  of  students  who  could  ansver  individual  items  or  sets  of  items 
correctly,  avoided  estimating  student  proficiency  distributions,  and  did  not 
make  individual  proficiency  measures  available  to  the  secondary  user.  The 
lack  of  individual  proficiency  measurements  encumbered  analyses  of  the 
relationships  between  proficiency  and  student  characteristics.  Regrettably, 
it  is  common  in  educational  surveys  to  carry  out  these  latter  analyses  with 
poorly  estimated  scores  for  individuals,  despite  the  demonstrable  invalidity 
of  their  results  (see  Goldstein,  1980). 

Recent  developments  in  item  response  theory,   in  statistical  estimation 
procedures,  and  in  methodologies  for  handling  missing  data  make  it  possible 
to  estimate  accurately  student  proficiency  distributions  and  their 
relationships  with  background  variables  from  complex,  sparse  sampling 
designs.     The  embodiment  of  these  advances,  the  derived  variables  called 
"plausible  values"  for  reading  and  writing,  were  constructed  to  yield 
consistent  estimates  of  such  population  characteristics  for  the  NAEP 
populations  as  a  whole,  and  for  the  subpopulations  defined  by  the  traditional 
NAEP  reporting  categories.     The  intricacies  and  expense  involved  in  obtaining 
optimal  estimates  from  such  a  complex  database  for  a  specific  analysis  may 
prove  prohibitive  to  most  secondary  analysts;  thus,   the  plausible  values 
mentioned  above  are  provided  for  exploratory  analyses.     Part  II  of  this 
report  provides  details  on  the  construction  and  properties  of  plausible 
values  and  caveats  on  their  use. 


ORGANIZATION  OF  THE  TECHNICAL  REPORT 

The  organizational  strategy  for  this  report  is  to  first  provide 
overviews  of  the  two  components  of  NAEP  described  herein,  design  and 
analysis.     These  overviews  direct  the  reader  to  chapters  where  details  are 
provided.     Each  chapter  begins  with  a  summary,   then  presents  a  detailed 
exposition  of  its  topic.     In  some  cases,  chapters  refer  to  appendices  or 
supplementary  documents  which  contain  even  more  detail.     This  strategy  has 
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been  adopted  to  aid  the  reader  in  reaching  areas  of  special  interest  The 
reader  who  wishes  only  a  summary  may  read  just  the  overviews  (Chapters  1  and 

^     The  chapters  are  separately  authored  and  differ  somewhat  in  style  and 
point  of  view.     In  most  cases,   the  person  most  responsible  for  the  activity 
was  assigned  the  writing  cask.     We  hope  that  the  chapters  can  be  read 
independently,  after  the  appropriate  overviews  are  read.     Although  we  have 
tried  to  cross-reference  where  necessary,   the  method  of  organization 
necessarily  results  in  some  redundancy  from  chapter  to  chapter 
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Chapter  1 


OVERVIEW  OF  PART  I:     THE  DESIGN  AND  IMPLEMENTATION 

OF  1986  NAEP 


Albert  E.  Beaton 
Educational  Testing  Service 


This  chapter  provides  a  description  of  the  1986  NAEP  design  and  an 
overview  of  the  processes  by  which  NAEP  evolved  from  the  planning  stage  into 
a  database  ready  for  analysis.     The  major  components  of  the  assessment  with 
few  details,  are  presented  here  with  references  to  the  appropriate  chapters 
m  Part  I  for  more  information.     Although  these  chapters  contain  most  of  the 
important  details  about  the  design  and  implementation  of  the  1986  assessment 
some  of  the  chapters  direct  the  reader  to  even  greater  detail  to  be  found  in' 
appendices  and  supplementary  documents.     The  report  is  organized  to  help  an 
interested  reader  first  to  locate  the  areas  of  his  or  her  interest    then  to 
study  those  areas  in  as  much  depth  as  necessary  to  understand  the  procedures 
and  considerations  involved  in  the  collection  of  NAEP  data.     From  this 
report,   it  is  hoped  that  the  reader  will  be  provided  with  a  comprehensive 
overview  of  the  1986  assessment  and  will  be  able  to  judge  the  quality, 
strengths,  and  weaknesses  of  the  data. 

This  part  of  the  technical  report  does  not  include  a  discussion  of  the 
procedures  used  in  data  analysis;  these  are  summarized  in  Chapter  7  (the 
overview  of  Part  II)  and  discussed  in  detail  in  Chapters  8  through  14  This 
report  does  not  include  the  substantive  results  of  the  1986  assessment  which 
are  published  separately  in  NAEP  cross -sectional  and  trend  reports  on  student 
achievement  in  individual  subject  areas.  * 

The  contents  of  this  chapter  are  as  follows: 

To  provide  background,  section  1.1  presents  the  NAEP 
assessment  schedule  from  the  first  year  of  data 
collection  in  1969  to  the  1986  assessment.  The 
assessments  in  progress  or  planned  through  1990  are  also 
mentioned. 


An  overview  of  the  NAEP  design  for  1986  is  presented  in 
section  1.2.     The  general  ETS  approach  to  the  design  of 
NAEP  is  covered  extensively  in  A  New  Design  for  a  New  Era 
(Messick,  Beaton,  &  Lord,  1983)  and  Implementing  the  New 
Design:  The  NAEP  1983-84  Technical  Report  (Beaton 
1987a). 

The  NAEP  1986  four- stage  stratified  random  sampling 
procedure  is  described  in  section  1.3.  Sampling  is 
described  more  fully  in  Chapter  3. 
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T'  e  development  of  the  objectives  and  items  for  different 
subject  areas  and  the  processes  by  which  they  were 
reviewed  is  summarized  in  section  1.4  with  more  detail 
provided  in  Chapter  2. 

The  assignment  of  the  NAEP  cognitive  and  background  and 
attitude  questions  to  booklets  is  presented  in  section 
1.5.     Detailed  information  is  presented  in  Chapter  4. 

The  field  administration  procedures,  including  the 
training  of  the  field  administrators,  attaining  school 
cooperation,  assessment  administration,  and  quality 
control  are  summarized  in  section  1.6;  more  description 
is  given  in  Chapter  5. 

The  flow  of  data  from  their  receipt  at  ETS  through  data 
entry,  professional  scoring,  and  entry  into  the  database 
in  final  form,  ready  for  analysis  is  summarized  in 
section  1.7  and  presented  in  detail  in  Chapter  6. 

The  data  collected  in  the  1986  assessment  have  been  prepared  for  public 
use  in  the  form  of  a  set  of  data  tapes,  documented  in  the  National  Assessment 
of  Educational  Progress  1985-86  Public-Use  Data  Tapes  Version  2.0  Users' 
Guide  (Rogers,  Kline,  Norris,  Johnson,  Mislevy,  Zwick,  Barone,  &  Kaplan, 
1988).     These  tapes  contain  the  data  available  for  the  sampled  students, 
their  teachers,  students  excluded  from  the  sample,  principals,  schools,  and 
computer  coordinators . 

1.1    ASSESSMENT  SCHEDULE 

The  coverage  of  assessments  through  1986  is  shown  in  Table  1.1,     As  this 
table  illustrates,   the  subject  areas  assessed  over  the  years  have  been  many 
and  varied,   including  not  only  reading,  writing,  mathematics,  and  science, 
but  many  other  areas  as  well--social  studies,  citizenship,  literature,  art, 
music,  and  career  development.     Many  subject  areas  have  been  re- assessed 
periodically  to  determine  trends  in  achievement  over  time.     Note  that,  by 
decision  of  the  Learning  Area  Committee  for  literature  (see  section  1.4),  the 
1986  literature  assessment  did  not  cover  the  same  material  as  that  of  1971; 
thus,  there  were  no  overlapping  items  and  analyses  of  trends  in  literature 
could  not  be  performed. 

Assessments  were  conducted  annually  through  1980,  but  budget 
restrictions  since  then  have  reduced  data  collection  to  a  biennial  basis. 
Since  its  inception,  NAEP  has  assessed  9-year-olds,  13-year-olds,  and  in- 
school  17-year-olds,  although  the  definitions  of  9-  and  13-year-olds  changed 
in  1986  (see  section  1.2).     Budget  restrictions  have  forced  NAEP  to  eliminate 
the  routine  assessment  of  out-of -school  17 -year-olds  and  young  adults. 
However,  in  1985  a  separately  funded  assessment  of  the  literacy  of  young 
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adults  was  administered.     The  results  have  been  published  in  Literacy: 
Profiles  of  Americans  Young  Adults.  Final  Report  (Kirsch  &  Jungeblut,  1986) 
This  survey  also  collected  a  small  sample  of  out-of -school  17-year-oids . 

Table  1.1  also  indicates  the  initiation  of  data  collection  by  grade  as 
well  as  by  age  in  1984,  a  practice  that  was  continued  in  the  1986  assessment. 

Assessments  through  1990  are  either  in  progress  or  in  the  planning 
stage.     The  1988  assessments  of  reading,  writing,  mathematics,  science, 
civics,  U.S.  history,  and  geography  are  in  progress.     Current  plans  call  for 
the  assessment  of  reading,  mathematics,  and  science  in  1990. 


1.2    THE  1986  NAEP  DESIGN 

The  1986  National  Assessment  was  designed  not  only  to  assess  student 
performance  in  1986  but  also  to  measure  changes  from  past  performance  of 
students  in  American  schools.     Because  new  age  definitions  and  ti-ies  of 
testing  were  introduced  into  the  1986  assessment,  the  design  w^^s  adjusted  to 
ensure  the  proper  measurement  of  any  changes  from  performance  in  earlier 
assessments  conducted  using  the  old  definitions.     This  section  describes  that 
design. 

The  desiga  of  the  1986  national  assessment  included  eight  samples  that 
differed  in  important  ways.     The  samples  can  be  classified  into  three 
different  types: 

Hain  NAEP  (spiral)  samples.     These  are  the  largest 
samples  and  incorporate  modifications  from  past 
assessments  in  population  definition  and  assessment 
technology.     These  samples  are  not  directly  comparable 
with  past  NAEP  data. 

Trend  (Bridge  A)  samples.     These  samples  employ  the 
population  definitions  and  assessment  technology  of  past 
assessments  and  thus  are  directly  comparable  with  past 
NAEP  data. 

Method  (Bridge  B)  samples.     These  samples  are  from  the 
same  populations  as  the  main  NAEP  samples  but  are 
measured  in  the  same  way  as  were  past  assessments.  Their 
purpose  is  to  measure  the  effect  of  the  modifications 
that  were  implemented  in  the  1986  assessment. 

The  following  sections  describe  the  attributes  of  these  samples.  The 
design  is  summarized  in  Table  1.2. 
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1.2.1    The  NAEP  Samples 


The  main  NAEP  samples,  denoted  (9),   (13),  and  (17)  in  Table  1.2 
incorporate  all  of  the  NAEP  modifications  introduced  in  1986: 

1)  Students  were  defined  as  being  age  9,  age  13,  or  age  17  if  they 
were  born  between  October  1  and  September  30  in  the  appropriate 
preceding  years,  that  is, 

Age  9:  October  1,  1976  to  September  30,  1977 
Age  13:  October  1,  1972  to  September  30,  1973 
Age  17:  October  1,  1968  to  September  30,  1979. 

This  definition  of  birthdates  is  of  the  same  form  as  NAEP  has  used 
in  past  assessments  for  17-year-olds  but  differs  for  9-  and  13- 
year-olds,  who  were  previously  defined  on  a  calendar-year  (January 
1  to  December  31)  basis.     This  modification  defines  the  three 
populations  in  an  equivalent  way;  consequently,   they  may  be 
considered  as  cohorts  who  were  born  four  years  apart. 

2)  Changes  in  age  definitions  for  9 -year-olds  and  13 -year-olds  also 
changed  their  corresponding  modal  grades  (the  grade  attended  by 
most  of  the  students  in  that  age  group)  Under  the  past  NAEP 
definitions,  at  the  time  of  assessment  most  of  the  9-year-olds  had 
been  in  4th  grade,  most  of  the  13 -year- olds  had  been  in  8th  grade, 
and  most  of  the  17-year-olds  had  been  in  11th  grade.     Under  the  new 
definitions,  most  of  the  9 -year-olds  were  in  3rd  grade  and  most  of 
the  13-year-olds  were  in  7th  grade.     Since  the  definition  of  the 
17-year-old  sample  did  not  change,   the  modal  grade  for  these 
students  remained  11th  grade.     This  modification  resulted  in  the 
grades  sampled  being  four  grades  apart  (grades  3,   7,  and  11)  rather 
than  irregularly  spaced  (grades  4,   8,   and  11)  as  in  the  past. 

3)  All  populations  were  defined  by  both  age  and  grade  levels.  That 
is,  a  student  was  eligible  for  assessment  if  he  or  she  was  either 
9-years-old  or  in  the  3rd  grade,  13-years-old  or  in  the  7th  grade, 
or  17-years-old  or  in  the  11th  grade.     Sampling  by  both  grade  and 
age,  or  "grade/age"  sampling,  was  introduced  in  the  1984 
assessment.     The  purpose  of  grade/age  sampling  was  to  permit  data 
to  be  analyzed  either  by  age,  as  in  the  past,  or  by  grade,  which  is 
often  preferred. 

4)  Students  were  all  assessed  near  the  end  of  the  school  year.  We 
refer  to  this  assessment  period  as  the  "spring"  even  though  the 
assessment  administration  started  on  February  17  and  continued 
until  May  2,  1986.  Assessing  all  ages  at  the  same  time  of  year 
allows  a  full  four  years  of  average  academic  growth  between  age 
populations . 
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5)  Students  were  assessed  in  reading,  mathematics,  science,  and 
computer  competence.     A  subset  of  the  grade  11/age  17  students  was 
assessed  in  U.S.  history  and  literature. 

6)  The  main  NAEP  sample  included  a  specially  funded,  separate  probe  of 
language  minority  students.  Some  language  minority  students  were 
assigned  a  separately  developed  booklet  with  a  section  of  NAEP 
reading  items,  a  section  of  NAEP  mathematics  items,  and  a  separate 
section  that  included  items  of  special  relevance  to  language 
minority  students.  This  study  is  reported  in  The  Educational 
Progress  of  Language  Minority  Children:     Findings  from  the  NAEP 
1985-86  Special  Study  (Baratz-Snowden,  Rock,  Pollack,  &  Wilder, 
1988) . 

7)  All  samples  were  administered  using  BIB  spiralling  (see  Messick, 
Beaton,  &  Lord,  1983;  Beaton,  1987a)  and  thus  used  printed 
instructions.     The  various  subject  areas  were  spiralled  together  so 
that  of  the  three  item  blocks  administered  to  each  student  a 
particular  student  might  receive,  for  example,  three  reading 
blocks;  some  combination  of  reading  blocks  and  blocks  from  other 
subject  areas;  or  no  reading  blocks  and  three  blocks  from  other 
subject  areas.     BIB  spiralling  was  introduced  in  the  1984 
assessment  of  reading  and  writing  and  was  used  for  assessments  of 
mathematics  and  science  for  the  first  time  in  1986. 


1.2.2    Trend  Samples 

The  several  design  changes  introduced  into  the  main  NAEP  samples  clearly 
make  comparisons  with  past  assessments  difficult,  and  so  several  other 
samples  were  collected  for  use  in  trends.     These  samples  were  defined  and 
administered  in  the  same  way  as  were  past  assessments.     These  samples  are 
denoted  (9a),   (13a),  and  (17b)  in  Table  1.2. 

The  samples  (9a)  and  (13a)  contained  9-year-olds  and  13-year-olds, 
respectively,  who  were  sampled  using  the  same  calendar-year  age  definition 
used  in  past  assessments.     The  ages  were  defined  as  follows  as  being  born  on 
or  between  the  following  dates: 

Age  9:  January  1,  1976  to  December  30,  1977 
Age  13:  January  1,  1972  to  December  30,  1973 
Age  17:  October  1,  1968  to  September  30,  1969. 

No  attempt  was  made  to  augment  the  age  9  sample  with  other  4th  graders  or  the 
age  13  sample  with  other  8th  graders  since  NAEP  did  not  traditionally  assess 
grades.     These  samples  were  assessed  in  reading,  mathematics,  and  science 
since  these  were  the  only  subject  areas  in  which  trend  data  were  available 
for  comparison.     Reading  was  measured  using  printed  administration,  matching 
the  procedures  from  the  1984  assessment,  with  which  the  data  were  to  be 
compared.     Mathematics  and  science  were  assessed  using  tape-recorded 
administration,  matching  the  procedure  used  in  the  several  previous 
assessments  with  which  performance  was  to  be  compared. 
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The  assessment  of  these  samples  was  scheduled  as  in  the  past,  with  the 
9-year-old  students  assessed  in  the  winter  and  the  13-year-old  students 
assessed  in  the  fall  of  the  school  year.     However,  the  age  9  sample  was 
assessed  only  during  January  1986,  a  somewhat  earlier  time  span  than  for  past 
assessments . 

Since  the  definition  of  age  and  the  time  of  assessment  were  not  changed 
for  the  17 -year-olds,  the  major  change  in  1986  for  this  sample  was  the  use  of 
printed  instructions  instead  of  a  paced  aural  presentation  by  tape  recorder. 
For  comparison  with  past  data,  a  method  sample,   (17b)  in  Table  1.2,  was 
appropriate.     The  sample  differed  from  the  main  NAEP  sample  (17)  in  that  the 
assessment  was  administered  using  a  tape  recorder  and  that  only  mathematics 
and  science  were  assessed.    A  bridge  sample  for  reading  was  not  necessary  for 
17 -year-olds  since  the  main  NAEP  sample  for  1986  was  similar  in  de&ien  to 
that  for  1984.  ^ 


1.2.3    Method  Samples 


The  NAEP  design  also  included  several  samples  for  measuring  the  effect 
of  changing  from  administration  by  tape  recorder  to  administration  by  print 
These  samples  are  denoted  (9b),   (13b),  and  (17b)  in  Table  1.2.     These  samples 
were  selected  from  the  same  populations  as  the  main  NAEP  samples;  thus,  their 
proficiencies  in  various  subject  areas  can  be  expected  to  be  the  same  as 
those  of  the  main  samples.     In  principle,  since  the  only  differences  between 
these  samples  and  the  corresponding  age  subsamples  of  the  main  NAEP  samples 
are  due  to  changing  from  tape-recorded  to  printed  administration  and  due  to 
sampling  error,  comparing  these  samples  can  allow  the  estimation  of  the 
effects  of  changing  methodology. 

Ages  were  defined  in  the  same  way  as  for  the  main  NAEP  sample,   that  is, 
the  students  would  be  eligible  for  selection  if  born  on  or  between  the 
following  dates: 

Age  9:  October  1,  1976  to  September  30,  1977 
Age  13:  October  1,  1972  to  September  30,  1973 
Age  17:  October  1,  1968  to  September  30,  1969. 

All  three  of  the  method  samples  were  assessed  only  in  mathematics  and 
science,  since  these  were  the  only  areas  that  changed  in  method  of 
administration  since  their  last  assessments.     The  samples  were  of  age  groups 
only,  not  grades,  since  past  assessments  used  age-only  sampling. 


1.3    THE  NAEP  SAMPLING  PLAN 

The  NAEP  populations  comprised  students  of  various  ages  and  grades  in 
the  50  states  and  the  District  of  Columbia.     Both  public  and  private  school 
students  were  sampled.     The  precise  subpopulations  varied  in  different 
samples.     In  the  main  NAEP  samples  and  the  samples  used  to  investigate  the 
effect  of  changes  in  administrative  procedures,  9-,  13-,  or  17-year-olds  we 
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defined  in  such  a  way  that  the  majority  of  them  were  in,  respectively,  3rd, 
7th,  and  11th  grades  (the  modal  grades).     In  trend  samples,  which  defined  age 
in  the  traditional  NAEP  manner,  the  majority  of  the  9 -year- olds  were  in  the 
4th  grade  and  the  majority  of  the  13 -year- olds  were  in  the  8th  grade. 

The  entire  NAEP  data  collection  effort  involves  a  four-stage  probability 
sample.     The  original  sampling  plan  for  NAEP  was  designed  by  the  Research 
Triangle  Institute  but  has  been  modified  by  Westat,  Inc.     The  four  stages  are 
summarized  briefly  below  and  in  somewhat  more  detail  in  Chapter  3.     For  a 
full  description  of  the  sampling  plan,  see  National  Assessment  of  Educational 
Progress- -17th  Year  Sampling  and  Weighting  Procedures,     Final  Report  (Burke, 
Braden,  Hansen,  Lago,  &  Tepping,  1987). 

Stage  1:     Primary  sampling  iinits.     In  the  first  stage  of  sampling,  the 
United  States  was  divided  into  geographical  units  comprising  counties  or 
groups  of  contiguous  counties  that  met  a  minimum  school  enrollment  size, 
The^^e  units,  called  primary  sampling  units  (PSUs)  ,  were  classified  into  12 
stratc ,  which  were  defined  by  region  (Northeast,  Southeast,  Central,  West), 
by  *ype  of  PSU  (MSA  [metropolitan  statistical  area],  non-MSA)  and  by  percent 
minority  (less  than  20  percent,  20  percent  or  more).    Ninety-four  PSUs  were 
selected  from  a  total  of  1,027.     Among  the  larger  PSUs,  34  were  selected  with 
certainty.    Within  each  major  stratum,  further  stratification  was  achieved  by 
ordering  tbc  noncertainty  PSUs  according  to  additional  socioeconomic 
charac:eLidt^cs.     Sixty  other  PSUs  were  selected  from  the  noncertainty  PSUs 
wit*h  piJobab'iity  proportional  to  size. 

Stage  2:     Sampling  schools.^    In  the  second  stage  of  sampling,  the  frame 
cotioisted  of  a  file  of  schools  obtained  from  Quality  Education  Data,  Inc.  The 
file  included  public,  private.  Catholic,  Bureau  of  Indian  Affairs,  and 
Department  of  Defense  schools,  listed  according  to  size  and  separately  for 
the  three  grade/age  groups,  within  each  of  the  94  PSUs.     High  minority 
schools  were  assigned  extra  weight  to  increase  the  probability  of  selection 
and  thereby  increase  minority  sample  sizes.     Schools  with  fewer  than  20 
estimated  grade/age  eligibles  were  assigned  lower  weights  because  of  the  high 
costs  of  administration.     The  method  of  assigning  weights  to  other  schools  is 
discussed  in  Chapter  3.     Schools  within  each  PSU  were  selected 
(systematically  with  random  start  and  without  replacement)  with  probability 
proportional  to  assigned  size. 

Stage  3:  Assignment  of  sessions  to  schools,  by  type.     In  the  third  stage 
of  sampling,  assignment  of  sessions  to  schools  was  done  separately  by  the 
three  types  of  sessions,  designated  "spiral",   "Bridge  A",  and  "Bridge  B," 
which  represent  separate  samples  of  the  population  of  students.  Schools 
selected  for  Bridge  A  samples  were  excluded  from  the  spiral  and  Bridge  B 
samples  because  the  Bridge  A  assessments  took  place  during  a  different  time 
of  year.     Except  in  the  case  of  a  smaller  school,  both  spiral  and  Bridge  B 
sessions  were  conducted  in  each  Spring- selected  school.  Smaller  schools  were 
assigned  randomly  to  either  spiral  or  Bridge  B  assessment. 

Stage  4:  Sampling  students.     In  the  fourth  stage  of  sampling,  a 
consolidated  list  of  all  grade-  and  age-eligible  students  was  established  for 
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each  selected  school.     A  systematic  selection  of  eligible  students  was  made 
and,  for  schools  in  the  spring  assessment,     students  were  assigned  to  spiral 
or  Bridge  B  sessions,  depending  on  whether  the  assessment  was  to  be 
administered  by  pencil  and  paper  (spiral)  or  by  tape  recorder  (Bridge  B) . 

Stage  4a:  Sampling  excluded  students.     Some  selected  students  were 
deemed  unassessable  by  school  authorities  because  they  had  limited  English 
language  proficiency,  were  judged  as  being  educable  mentally  retarded,  or 
were  functionally  disabled.     In  these  cases,  an  excluded  student 
questionnaire  was  filled  out  by  the  school  staff  listing  the  reason  for 
excluding  the  student  and  providing  some  background  information. 

Stage  4b:  Sampling  teachers.     The  teacher  questionnaire  was  administered 
to  the  teachers  of  a  subsample  of  the  students  sampled  for  spiral  sessions. 
The  purpose  of  this  sample  was  to  estimate  the  number  (proportion)  of 
students  whose  teachers  had  various  attributes,  not  the  percent  of  the 
teacher  population  who  had  various  attributes.     Therefore,  statements  such  as 
"20  percent  of  students  have  teachers  who  have..."  are  appropriate  in 
discussing  teacher  questionnaire  data,  but  statements  such  as  "20  percent  of 
teachers  have..."  are  not. 

One  subject  area  was  designated  for  each  spiral -allocated  school.  For 
grade  3/age  9,  the  subject  area  was  always  reading.     For  grade  7/age  13,  the 
subject  area  could  have  been  reading,  mathematics,  or  science.     For  \e 
11/age  17,  the  subject  area  could  have  been  reading,  mathematics,  science,  or 
U.S.  history.     Teachers  of  only  one  such  subject  area  were  sampled  in  a 
school . 

For  each  spiral  session  in  each  school,  a  subsample  of  students  was 
selected  and  the  school  coordinator  was  asked  to  identify,  for  each  selected 
student,  the  teacher  in  the  designated  subject  area  who  was  teaching  the 
student.     (For  reading,  English  or  language  arts  teachers  were  selected.) 
These  teachers  were  asked  to  complete  the  teacher  questionnaire.     Since  a 
number  of  students  may  have  had  the  same  teacher,  and  some  teachers  did  not 
complete  the  questionnaire,  the  number  of  students  in  the  subsample  for  whom 
teacher  information  is  available  is  not  the  same  as  the  number  of  teachers 
who  completed  questionnaires  in  a  given  school. 

Stage  4c:  The  principal,  school,  and  computer  questionnaires.  A 
principal  questionnaire,  distributed  to  each  sampled  school  by  Westat  prior 
to  the  assessment,  was  used  by  Westat  to  obtain  both  an  up-to-date  estimate 
of  grade/age-eligible  students  and  information  on  minority  enrollment. 

The  school  characteristics  and  policies  questionnaire  was  distributed  in 
every  sampled  school.     The  school  characteristics  and  policies  questionnaire 
was  mailed  to  ^he  school  by  Westat  prior  to  the  assessment,  collected  by  the 
Westat  supervisor,  and  returned  to  ETS. 

In  every  school  selected  for  the  spiral  sample,  the  school's  computer 
coordinator,  if  there  was  one,  was  asked  to  fill  out  a  computer  coordinator 
questionnaire.     This  questionnaire  was  administered  at  all  grade/age  levels. 
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1.4    DEVELOPMENT  OF  NAEP  ASSESSMENT  ITEMS 


The  1986  NAEP  assessed  the  performance  of  students  in  reading, 
mathematics,  science,  and  computer  competence.     NAEP  also  assessed  U.S. 
history  and  literature  at  grade  11/age  17.  In  addition,  a  large  number  of 
background  and  attitude  questions  were  asked  of  the  students  and  information 
was  collected  from  principals  and  teachers. 

The  development  of  items  was  supervised  by  Learning  Area  Committees,  one 
of  which  was  appointed  by  the  Assessment  Policy  Committee  for  each  curriculum 
area.     Each  Learning  Area  Committee  developed  a  set  of  objectives  for  its 
area;  these  objectives  represented  a  consensus  of  what  students  at  each 
grade/age  level  should  know  and  be  able  to  do.     Besides  items  developed  to 
assess  academic  achievement  (cognitive  items) ,  the  Learning  Area  Committees 
also  developed  items  to  investigate  student  attitudes,  experiences,  and 
interests  (noncognitive  items).     In  addition,  a  common  core  of  items  was 
developed  to  collect  data  about  a  student's  personal  and  family  background. 

All  items  underwent  extensive  reviews  by  subject-matter  and  measurement 
specialists,  as  well  as  careful  scrutiny  to  eliminate  any  potential  bias  or 
lack  of  sensitivity  to  particular  groups.     The  items  used  in  each  assessment 
have  been  made  available  to  anyone  interested  in  studying  or  using  them 
provided  that  they  agree  not  to  make  the  items  public.     The  items  have 
traditionally  been  kept  secure  for  use  in  future  assessments  for  the 
examination  of  trends  over  time . 

All  assessment  areas  except  U.S.  history  and  literature  contained 
multiple-choice,  short  open-ended,  and  long  open-ended  items.     The  open-ended 
items  were  professionally  scored.     The  professional  scoring  process  is 
described  in  Chapter  6.2. 

Details  on  item  development  are  given  in  Chapter  2 . 


1.5    THE  NAEP  ASSESSMENT  BOOKLETS 

Booklets  containing  the  items  were  assigned  to  the  student  samples  that 
were  selected.  The  BIB  spiralling  design  of  NAEP  determined  the  way  in  which 
the  assessment  booklets  were  organized  and  constructed.     The  type  of  booklet 
assigned  depended  on  whether  the  student  was  in  the  spiral  or  bridge  samples. 
A  detailed  discussion  of  this  topic  can  be  found  in  Chapter  5. 

1.5.1    Spiral  Sample  Booklets 

The  main  NAEP  sample  was  assessed  using  BIB  spiralling  (see  Messick, 
Beaton,  &  Lord,  1983;  Beaton,  1987a). 

The  targeted  sample  size  was  for  2,000  students  to  respond  to  each  item 
at  each  age  or  grade  level  in  the  spiral  sample;  this  target  implied  a  sample 
of  2,600  at  each  grade/age. 
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The  BIB-spiral  sample  was  created  to  meet  the  design  goal  of 
facilitating  the  estimation  of  intercorrelations  or  other  statistics  among 
the  assessment  items.    Using  a  BIB-spiral  design,  a  large  ntimber  of  booklets 
was  created  in  such  a  way  that  each  pair  of  items  was  administered  to  a 
randomly  equivalent  subsample  of  students  while  maintaining  the  goal  of  2,000 
students  for  each  item  at  both  age  and  grade  levels. 

Briefly,  BIB  spiralling  was  implemented  in  1986  as  follows: 

The  items  from  the  different  subject  areas  (e.g.,  reading,  mathematics) 
were  sorted  into  units  called  blocks,  which  were  designed  to  take  sixteen 
minutes  for  the  older  students  to  complete.     For  the  grade  3/age  9  students, 
the  subject  area  items  were  expected  to  take  thirtec minutes.  Altogether, 
there  were  23  blocks  of  items  used  at  grade  3/age  9,  30  blocks  at  grade  7/age 
13,  and  42  blocks  at  grade  11/age  171evel.     Some  blocks  were  administered  at 
more  than  one  age  and  grade. 

Th^se  blocks  were  then  assembled  into  booklets.     Each  booklet  contained 
a  commoT.  block  and  three  blocks  containing  subject  area  items.     The  common 
block  contained  only  background  and  attitude  questions.     In  a  completely 
balanced  incomplete  design,  the  subject  area  blocks  would  have  been  placed  in 
booklets  in  such  a  way  that  each  pair  of  blocks  appeared  together  in  one  and 
only  one  booklet.     However,  as  the  number  of  blocks  increases,  the  number  of 
booklets  required  becomes  large  and  thus  the  booklet  design  called  for 
complete  BIB  spiralling  within  each  subject  area,  which  is  easily  manageable, 
and  an  incomplete  block  design  between  subject  areas.     (The  history  and 
literature  books  at  grade  11/age  17  were  for  a  special  probe  and  were 
spiralled,  but  not  in  such  a  way  that  each  pair  of  blocks  were  assigned  to 
some  student.)  For  the  main  NAEP  sample,  46  booklets  were  printed  for  the 
grade  3/age  9  sample,  62  for  the  grade  7/age  13  sample,  and  90  for  the  grade 
11/age  17  sample. 

The  booklets  for  the  main  NAEP  sample  were  then  spiralled  and  placed 
into  bundles.     The  spiralling  involved  interleaving  the  booklets  in  a  random 
sequence  so  that  each  booklet  would  occur  the  appropriate  number  of  times  for 
each  item  to  be  administered  to  approximately  2,600  student  in  a  grade/age 
sample.     The  bundle  size  was  29  booklets,  which  was  intended  to  be  large 
enough  for  most  assessment  sessions  with  a  few  books  left  over.     The  bundles 
were  designed  so  that  each  booklet  was  at  the  top  of  a  bundle,   in  each 
position  in  the  middle  of  a  bundle,  and  at  the  end  of  a  bundle  an  equal 
number  of  times . 

With  BIB  spiralling,  the  assessment  booklets  are  assigned  to  students  in 
the  same  order  as  the  booklets  are  bundled  so  that  different  students  in  the 
same  assessment  session  are  asked  to  respond  to  different  items.  With 
spiralling,  the  instructions  to  the  students  and  the  items  themselves  must  be 
read  by  the  student  from  his  or  her  booklet  since  administration  using  a  tape 
recorder  would  be  unmanageable  with  more  than  one  type  of  booklet  in  an 
assessment  session.     One  of  the  advantages  of  spiralling  is  an  increase  in 
sampling  efficiency. 


1.5.2  Bridge  A  and  Bridge  B  Sample  Booklets 


The  Bridge  A  and  Bridge  B  booklets  could  not  be  BIB  spiralled  since  they 
were  intended  for  administration  by  tape  recorder  as  in  the  past  assessments, 
with  the  exception  of  the  reading  block  in  each  Bridge  A  booklet.  Several 
special  booklets  were  created  for  these  bridge  samples. 

Three  booklets  were  printed  for  Bridge  A  at  grade  3/age  9  and  three  more 
for  grade  7/age  13.     There  was  no  separate  Bridge  A  sair.ple  at  grade  11/age 
17.     Each  booklet  contained  one  block  of  reading,  one  of  mathematics,  and  one 
of  science.     A  different  reading,  mathematics,  and  science  block  was  used  in 
each  booklet,  thus  three  blocks  from  each  of  these  subject  areas  were 
administered.     The  order  in  which  the  subject  area  blocks  were  presented  was 
changed  in  the  three  booklets.     See  Chapter  4  for  details. 

In  each  Bridge  A  session,  all  students  were  administered  the  same  one  of 
the  three  available  booklets.  The  mathematics  and  science  blocks  were 
administered  using  a  tape  recorder,  which  gave  instructions  and  presented  the 
questions  and  optional  answers  aurally.     The  tape  recorder  was  turned  off 
when  the  reading  block  was  presented  because  reading  had  been  bridged  in 
1984. 

Two  booklets  were  printed  for  the  Bridge  B  sample  at  each  of  the  three 
grade/age  levels.     Only  mathematics  and  science  were  assessed,  since  the 
effect  of  changing  from  a  tape  recorded  to  printed  administration  had  already 
been  examined  for  reading.     Each  booklet  contained  either  two  mathematics  and 
one  science  block  or  one  mathematics  and  two  science  blocks.  Three 
mathematics  and  three  science  blocks  were  used  altogether.  In  each  Bridge  B 
session,  all  students  were  administered  the  same  one  of  the  two  appropriate 
booklets.     Both  booklets  were  administered  using  a  tape  recorder. 


1.5.3    Other  Assessment  Instruments 

The  excluded  student  questionnaire  was  developed  and  used  for  the  first 
time  in  the  1984  assessment.     It  was  designed  to  gather  more  information 
about  particular  reasons  for  which  a  student  was  excluded  and  some  basic 
characteristics  of  the  student,  such  as  race,  age,  etc. 

The  teacher  questionnaire  was  also  developed  and  used  for  the  first  time 
in  1984.     It  was  designed  to  gather  information  about  the  attributes  of  the 
teachers  of  the  students  in  the  sample  and  to  gather  some  information  about 
the  curricula  and  teaching  methods  in  the  classroom.     The  questionnaires  for 
the  teachers  at  all  grade/age  levels  had  different  sections  for  reading, 
matnematics,  and  science  curricula  and  teachers  of  grade  11/age  17  students 
had  questions  about  U.  S.  history  and  literature  curricula. 

The  school  characteristics  and  policies  questionnaire  was  distributed  to 
each  participating  school  to  be  completed  by  either  the  school's  principal  or 
another  person  familiar  with  data  concerning  enrollment,  facilities, 
curricula  and  staff  development. 
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A  questionnaire  was  also  administered  to  each  school's  computer 
coordinator,   if  the  school  had  ^ne.     This  questionnaire  included  questions 
about  subjects  in  which  computer-aided  instruction  was  used,  computer  courses 
and  subjects  taught,  and  computer  resources. 

More  information  about  the  items  and  instruments  can  be  found  in  Chapter 


1.6    NAEP  FIELD  ADMINISTRATION 

Wcstat  was  responsib/e  for  field  administration.     The  process  began  with 
the  development  of  necessary  materials  and  a  field  organization.  Materials 
were  developed  for  training,  contacting  the  schools,  sampling,  and  process 
control.     The  field  organization  consisted  of  district  supervisors  and 
exercise  administrators.     Westat  trained  the  district  supervisors,  who  in 
turn  trained  the  exercise  administrators. 

Gaining  school  cooperation  was  primarily  the  responsibility  of  Westat, 
with  considerable  support  from  NAEP  staff.     ETS  first  contacted  the  Chief 
State  School  Officers,  informing  them  that  schools  within  their  states  had 
been  selected  for  NAEP.     Later,  mailings  and  materials  were  sent  to  the  Chief 
State  School  Officers,  school  district  superintendents,  and  private  school 
officials.     Meeting  arrangements  were  then  established  by  telephone  and 
contact  forms  were  filed  with  Westat.     Westat  district  supervisors  then 
scheduled  and  conducted  introductory  meetings . 

Westat  administered  the  assessment  in  the  field  primarily  through  the 
work  of  district  supervisors.     District  supervisors  had  many 
responsibilities,  including  drawing  the  sample  of  students,  completing 
assessment  reporting  forms,  making  final  arrangements  for  the  assessments, 
supervising  exercise  administrators,  distributing  and  collecting  other  data 
forms  and  questionnaires,  and  editing,  boxing  and  shipping  assessment 
materials . 

The  spiral  and  Bridge  B  samples  were  assessed  between  February  17  and 
May  2,  1986  at  all  grade/age  levels.     The  grade  3/age  9  students  in  the 
Bridge  B  sample  were  assessed  between  January  6  and  January  31,  1986.  For 
Bridge  A  samples,  the  grade  7/age  13  students  were  assessed  between  November 
4  and  December  4,  1985. 

An  assessment  session  was  expected  to  last  approximately  one  hour.  At 
the  older  grade/age  levels,  the  students  were  allowed  six  minutes  for  the 
common  block  of  questions  and  16  minutes  for  each  block  of  subject  matter 
items  and  background  and  attitude  questions.     The  grade  3/age  9  sample  was 
read  the  common  block  questions  aloud  and  given  15  minutes  to  complete  that 
section.     Then,  they  were  given  13  minutes  to  respond  to  the  items  in  each  of 
the  three  subsequent  blocks. 

Both  \yestat  and  ETS  participated  in  the  quality  control  and  evaluation 
of  the  field  administration.     There  were  two  specifically  designed  quality 
control  and  evaluation  studies  of  the  field  effort.     The  first,  and  most 
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intensive,  primarily  focussed  on  quality  control  and  involved  on-site  visits 
by  Westat  and  ETS  staff  to  verify  the  sampling  and  to  observe  the  supervisors 
and  exercise  administrators  as  they  conducted  assessments.     The  second  study, 
an  evaluation,  was  a  telephone  survey  of  a  10-percent  sample  of  schools. 
This  survey  took  place  after  the  field  period  had  ended  and  all  assessment 
activities  had  been  completed  in  the  schools. 

Field  administration  is  discussed  in  detail  in  Chapter  5. 


1.7     DATABASE  CONSTRUCTION 

Westat  shipped  the  assessment  booklets  from  the  field  to  ETS  for  entry 
into  computer  files,  checking,  and  forming  the  database.     Careful  checking 
assured  that  all  data  from  the  field  were  received.    The  data  then  went 
through  extensive  processing,  outlined  in  Chapters  6  and  6.1. 

Since  both  machine  readable  (scannable)  and  nonscannable  instruments 
were  used,  the  "intelligent"  data  entry  system  developed  for  the  1984 
assessment  was  used  as  well  as  standard  document  reading  technology.  These 
computer  programs  not  only  received  the  input  data  but  also  checked  them  for 
consistency  among  the  many  different  booklets,  blocks,  and  formats.  The 
program  assured  that  all  entered  values  of  each  variable  were  within  the 
range  of  possible  values.     The  entry  and  editing  of  materials  is  r.iscussed  in 
Chapters  6.3  and  6.4. 

Many  items  in  each  subject  area  (reading,  mathematics,  sci:jnce,  and 
computer  competence)  required  open-ended  responses  and  had  to  he 
professionally  scored.  Professional  scoring  is  discussed  in  Chapter  6.2. 

Extensive  quality  control  checks,  described  in  Chapter  6.5,  were 
instituted  to  assure  correspondence  between  what  had  been  written  in  the 
booklet  and  what  appeared  in  the  database.     A  random  sample  of  each 
assessment  booklet  and  questionnaire  was  selected  from  the  computer  file  and 
checked  against  the  original  document.     The  database  was  determined  to  be 
extraordinarily  error- free. 

The  construction  of  the  database  and  public -use  data  tapes  are  described 
in  more  detail  in  Chapters  6.6  and  6.7. 
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DEVELOPING  THE  1986  NATIONAL  ASSESSMENT  OBJECTIVES,  ITEMS, 
AND  BACKGROUND  QUESTIONS 

Ina  V.  S.  Mullis,  Walter  MacDonald,  and  Nancy  A.  Mead 
Educational  Testing  Service 


2 . 1  INTRODUCTION 


The  1986  assessment  was  the  most  ambitious  ever  conducted  by  NAEP, 
including  an  extremely  complex  design,  modifications  in  sampling  and 
administration  procedures,  substantial  numbers  of  cognitive  items  for  six 
subject  areas,  and  questionnaires  for  students,  teachers,  and  school 
administrators.     The  development  process  began  in  the  fall  of  1983  when  the 
Assessment  Policy  Committee  determined  the  subjects  to  be  assessed.  In 
accordance  with  the  newly  granted  ETS  proposal  for  NAEP,  the  Assessment 
Policy  Committee  identified  four  curriculiim  areas  to  be  assessed  in  1986: 


Reading.     This  area  was  highlighted  in  the  ETS  proposal  for  inclusion  i 
each  biennial  NAEP  assessment.     Because  reading  is  central  to  proficiency  in 
other  subject  areas,  levels  of  reading  achievement  are  likely  to  serve  as  an 
overall  barometer  of  educational  progress  in  our  country.     Also,  with  the 
BIB-spiral  design,  assessment  of  one  subject  in  common  across  various  years 
can  be  used  to  link  results  from  year  to  year,  as  well  as  to  compare  the 
achievement  levels  of  sets  of  birth-year  cohorts  followed  in  the  NAEP 
assessments . 


Mathematics.     The  NAEP  legislation  in  place  through  1988  required  that 
NAEP  collect  and  report  at  least  once  every  five  years  data  assessing  the 
performance  of  students  at  various  age  or  grade  levels  in  each  of  the  areas 
of  reading,  writing,  and  mathematics.     This  requirement  was  satisfied  for 
reading  and  writing  in  the  1984  assessment;  however,  because  mathematics  had 
not  been  assessed  since  1981-82,  it  had  to  be  assessed  in  1986. 
Additionally,  the  decision  to  assess  mathematics  made  excellent  sense  because 
it  is  an  important  subject  area  and,  especially  considering  the  timing  of 
previous  mathematics  assessments,  a  1986  assessment  would  provide  valuable 
information  about  the  performance  of  birth-year  cohorts  of  students. 


Science.     Science  is  another  important  curriculum  area  that  is  currently 
of  much  inte^3st  and  concern  to  the  economic  well-being  of  our  country. 
Although  a  special  science  assessment  using  NAEP  materials  and  data 
collection  procedures  had  been  funded  by  the  National  Science  Foundation  in 
1981-82,  this  subject  had  not  been  assessed  by  NAEP  £er  se  since  1976-77. 


Computer  Competence,     It  was  noted  that  the  BIB -spiral  design  used  in 
1986  was  especially  well  suited  to  supporting  thematic  assessments,  and  that 
with  a  computer  component,  the  1986  assessment  would  provide  a  great  deal  of 
information  about  students'  understanding  of  technology.     Thus,   in  keeping 
with  the  desire  to  provide  baseline  information  about  achievement  in  this  new 
and  burgeoning  school  subject  and  to  develop  an  assessment  with  a  unified 
theme,  the  Assessment  Policy  Committee  selected  assessment  of  computer 
competence  as  the  fourth  subject  area. 

In  concluding  their  deliberations  about  the  focus  of  the  1986 
assessment,  the  Assessment  Policy  Committee  also  directed  staff  to 
concentrate  on  the  development  of  materials  measuring  higher-order  thinking 
skills . 


2.2  ASSESSMENT  POLICY  COMMITTEE  APPROVAL  OF  ASSESSMENT  DEVELOPMENT 
PROCEDURES 

In  addition  to  selecting  the  subject  areas  to  be  assessed  in  1986  and 
dixocting  the  staff  to  focus  on  higher-order  thinking  skills,  the  Assessment 
Policy  Committee  asked  staff  to  document  the  procedures  that  would  be  used  in 
developing  the  1986  assessment  materials  and  to  present  these  to  the 
committee  for  their  approval.     This  occurred  at  the  February  24-25,  1984 
meeting  of  the  Assessment  Policy  Committee,  and  the  procedures  follow,  as 
approved. 

2.3  SETTING  OBJECTIVES 

The  procedures  followed  for  determining  the  objectives  to  be  measured  in 
each  curriculum  area  in  1986  were  essentially  those  followed  by  NAEP  in  the 
past : 

1)  The  objectives  used  in  the  previous  assessment  were 
mailed  to  about  25  specialists  for  their  review, 
comments;  and  suggestions.     No  constraints  were  placed  on 
this  activity  and  we  asked  for  candid,  critical 
reactions.     The  individuals  involved  in  this  process 
tended  to  be  educators  and  specialists  in  the  field  and 
were  selected  to  represent  differing  points  of  view, 
geographical  locations,  backgrounds,  and  constituencies. 
We  sought  advice  from  a  wide  range  of  sources  for 
recommendations  for  this  activity. 

2)  Learning  Area  Committees  were  established  to  help  guide 
assessment  development  procedures  within  subject  areas. 
As  with  reviewers,  the  members  of  each  committee  were 
selected  witn  great  care  to  represent  differing 
perspectives  and  backgrounds.     These  21  committee  members 
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(5-6  per  subject  area)  worked  closely  with  NAEP  staff  in 
developing  the  1986  assessment. 

3)  Comments  from  the  initial  objectives  review  were 
synthesized  and  used  as  input  for  the  first  Learning  Ares 
Committee  meeting,  a  combined  meeting  of  all  four 
committees  held  January  27-29,  1984  (see  section  2.6). 

4)  The  first  assignment  of  the  respective  Learning  Area 
Committees  was  to  review  and  revise  the  broad  educational 
objectives  for  each  subject  area  based  on  their  personal 
feelings  and  the  comments  of  the  previous  reviewers. 

5)  The  new  edition  of  the  objectives  was,   in  turn,  mailed  to 
practitioners  from  around  the  country.     These  individuals 
are  school  administrators  and  teachers,  as  well  as 
teacher  trainers  who  live  and  work  in  the  practical 
educational  environments.     Their  task  was  to  review  these 
objectives  from  the  pcint  of  view  of  what  seems 
reasonable  and  practical.     Depending  upon  the  results  of 
that  review,   the  objective    were  redrafted  with  the 
participation  of  the  Lear  ang  Area  Committee  members  and 
others,  as  necessary. 

6)  The  revised  objectives  were  mailed  to  a  number  of  members 
of  the  lay  public  for  their  reactions  and  opinions.  As 
with  earlier  steps  in  the  objectives  development  process, 
care  was  taken  to  be  certain  that  appropriate  minority 
group  representatives  were  included  to  assure  proper 
attention  to  these  sensitivities. 

7)  Further  modifications  of  the  objectives  were  made  as 
necessary. 

8)  The  Learning  Area  Committees  completed  the  final  review 
of  the  product. 

9)  The  objectives  were  published,  printed,  and  made 
available  for  national  distribution. 


While  the  process  described  above  may  seem  tedious  and  detailed,  it 
seems  an  appropriate  series  of  steps  to  ensure  interested  parties  the 
opportunity  to  participate  in  the  objectives  development  process  and  to 
express  opinions.     (The  400  consultants  involved  in  developing  the  1986 
objectives  are  listed  at  the  end  of  this  chapter.) 
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2.4    PROCEDURES  FOR  DEVELOPING  THE  ITEMS 


A  carefully  developed  and  tested  series  of  steps,  essentially  those 
followed  in  the  past  by  NAEP,  were  used  to  create  test  items  that  reflect  the 
objectives  and  that  measure  achievements  related  to  them: 

1)  Each  Learning  Area  Committee  and  the  staff  of  NAEP 
determined  what  specific  aspects  of  the  objectives  could 
be  measured.     Each  respective  committee  made 
recommendations  about  priorities  for  the  assessment  and 
types  of  items  to  be  developed.     A  group  discussion 
regarding  measuring  higher- order  reasoning  skills  across 
subject  areas  was  also  conducted. 

2)  The  staff  then  drafted  a  development  and  analysis  plan 
delineating  the  steps  to  take  to  create  items  that  would 
produce  or  generate  the  data  essential  to  answer 
questions  the  assessment  was  designed  to  address. 

3)  The  plan  was  reviewed  by  the  staff,  the  Learning  Area 
Committees,  and  other  outside  reviewers  to  provide  an 
opportunity  to  critique  and  to  assure  that  no  important 
points  were  missed. 

4)  The  existing  pool  of  items  to  be  used  to  measure  change 
from  previous  assessments  (trend  items)  was  reviewed  in 
detail . 

5)  Item  specifications  were  then  developed  and  prototype 
items  were  created  to  reflect  the  type  of  questions  that 
had  been  suggested.     Trend  items  were  selected. 

6)  Item  writers  with  skills  and  experience  in  creating  items 
according  to  specifications  were  identified  both  from 
inside  and  beyond  ETS  and  scheduled  for  item  development 
tasks . 

7)  Newly  created  items  were  reviewed  and  revised  by  staff 
and  external  reviewers. 

8)  Further  language  editing  and  sensitivity  reviews  were 
conducted  according  to  ETS  quality  control  procedures. 

9)  Field  test  materials  were  prepared,   including  the 
materials  necessary  to  secure  clearance  by  the  Office  of 
Management  and  Budget  (0MB) . 

10)  The  field  test  was  conducted  with  a  representative  group 
of  students. 

11)  Field  test  booklets  were  scored  and  the  results  analyzed. 
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12)  Based  on  these  analyses  and  the  results  of  the  pilot 
testing,  items  were  revised  or  modified  and  re-edited. 
They  once  again  went  through  an  ETS  sensitivity  review. 

13)  With  the  help  of  staff  and  outsidt  reviewers,  the 
Learning  Area  Committee  selected  the  Items  to  include  in 
the  assessment. 

14)  Items  were  assembled  into  "blocks"  (14-minute  mini- 
tests)  according  to  statistical  guidelines  established  at 
the  beginning  of  the  process. 

15)  After  a  final  review  and  check  to  assure  that  each 
assessment  booklet  and  each  block  therein  met  the  overall 
guidelines  for  the  assessment,  the  booklets  were  typeset 
and  printed. 


2.^5    THE  FOUNDATIONS  OF  LITERACY  PROJECT 

The  assessments  of  basic  knowledge  in  U.S.  history  and  literature  were 
conducted  by  the  Educational  Excellence  Network  in  conjunction  with  NAEP 
with  support  from  the  National  Endowment  for  the  Humanities.  Because 
students  cannot  build  the  conceptual  understandings  necessary  for  reasoned 
thought  and  communication  without  knowing  basic  facts  within  these  curriculum 
areas,  the  Educational  Excellence  Network  obtained  funding  from  the  National 
Endowment  for  the  Humanities  to  collect  baseline  data  for  educators, 
policymakers,  curriculum  builders,  scholars,  and  parents  to  appraise  the 
extent  to  which  the  next  generation  of  Americans  possessed  the  rudimentary 
knowledge  that  forms  the  foundations  of  literacy. 

Although  pleased  to  receive  secondary,  funding  to  conduct  additional 
assessments,  the  Assessment  Policy  Committee  (APC)  gave  serious  consideration 
to  the  advantages  and  disadvantages  of  using  NAEP  as  the  vehicle  for  such  a 
knowledge  probe.     Previous  NAEP  assessments  of  social  studies,  literature, 
and  reading  had  included  knowledge  questions,  but  this  was  the  first  time' 
that  assessments  would  focus  exclusively  on  students'  basic  knowledge  of 
American  history  and  their  familiarity  with  the  major  authors,  themes,  and 
characters  of  Western  literature.     However,  the  APC  acknowledged  a  growing 
national  concern  that  a  number  of  young  Americans  lacked  rudimentary 
knowledge  of  U.S.  history  and  literature  and  that  systematic  information 
about  the  acquisition  of  this  knowledge  would  be  quite  beneficial. 
Therefore,  they  approved  NAEP  involvement  in  the  Foundations  of  Literacy 
project,  provided  that  the  assessments  be  developed  and  reported  using  the 
in-place  procedures  already  approved  by  the  committee  and  outlined  above. 
Thus,  NAEP  also  developed  materials  to  conduct  special  probes  assessing  the 
basic  knowledge  of  students  in  U.S.  history  and  literature.    Administered  in 
the  spring  of  1986,  these  assessments  were  given  only  to  nationally 
representative  probability  samples  of  17-year-olds  and  llth-grade  students, 
rather  than  to  all  three  grade/age  levels  included  for  the  other  four  subject 
areas. 
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2.6    THE  INITIAL  MEETING  OF  THE  LEARNING  AREA  COMMITTEES 


Although  detailed  previously,   it  should  be  noted  that  the  1986  NAEP 
development  process  was  governed  by  several  major  considerations: 

1)  As  outlined  in  the  ETS  proposal  for  the  administration  of 
the  NAEP  grant,   the  development  of  objectives  and  items 
for  each  curriculum  area  would  be  guided  by  a  Learning 
Area  Conmiittee. 

2)  As  specified  in  the  legislation,  the  objectives  would  be 
developed  through  a  consensus  process  involving  subject 
matter  experts,  school  administrators,  teachers,  and 
parents,  and  the  items  would  be  carefully  reviewed  for 
potential  bias. 

3)  The  ETS  Standards  for  Quality  and  Fairness  (ETS,  1983) 
describe  particular  procedures  and  reviews  for  all 
materials  developed  at  ETS. 

4)  All  NAEP  items  must  be  submitted  to  a  complex  0MB 
clearance  process  and  all  publications,  including 
objectives  booklets,  submitted  for  DERI  review. 

Because  of  NAEP's  goal  to  unify  the  four  subject  area  assessments 
designated  by  the  Assessment  Policy  Committee  through  the  themes  of 
technology  and  higher-order  thinking  skills,   the  development  of  the 
assessment  frameworks  for  all  four  areas- -reading,  mathematics,  science,  and 
computer  competence- -was  initiated  at  a  combined  meeting  of  the  four  Learning 
Area  Committees.     Held  in  January  of  1984,  this  combined  meeting  gave  NAEP 
staff  the  opportunity  to  welcome  these  important  committees,  explain  their 
general  task,  present  some  thoughts  about  threads  of  higher- order  thinking 
that  could  run  across  assessments,  and  describe  the  1986  design.     After  the 
plenary  sessions,  each  committee  retired  to  develop  the  general  framework  for 
the  assessment  in  their  subject  area.     The  meeting  concluded  with  a  final 
general  session  where  each  group  presented  its  framework  and  answered 
questions , 

2.7    DEVELOPING  THE  READING  ASSESSMENT 
2.7.1  Objectives 

The  objectives  for  the  1986  reading  assessment  (see  NAEP,  1987a)  were 
formulated  to  reflect  an  interactive  view  of  reading  encompassing  the  type  of 
material  being  read,   the  reader's  purpose,  and  the  background  knowledge  that 
the  reader  brings  to  the  reading  experience.     The  objectives  as  such  were  not 
limited  to  particular  grade/age  levels,  since  all  readers  past  the  earliest 
period  of  learning  to  read  engage  in  all  of  the  activities  included. 
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Four  objectives  were  drafted  by  the  conunittee,  with  these  and  their 
descriptions  reviewed  by  outside  constituencies  and  revised  as  necessary. 
The  members  of  the  Learning  Area  Committee  remained  involved  throughout  the 
review  and  revision  process.    While  objectives  defined  from  such  a  consensual 
process  cannot  specifically  advance  either  a  single  theoretical  framework  or 
the  views  of  any  one  individual,  they  do  represent  the  thinking  of  a  broad 
cross-section  of  individuals  who  are  deeply  concerned  with  reading  in  our 
schools . 

NAEP's  1984  reading  objectives  reflected  the  view  that  the  processes  of 
comprehension  and  the  extension  of  comprehension  through  interpretation  and 
analysis  have  a  place  in  reading  all  kinds  of  text.     Building  on  this  view, 
the  first  objective  underlying  the  1986  assessment  was  Comprehends  What  is' 
Read.     This  objective  included  comprehension  of  a  wide  variety  of  materials 
as  well  as  materials  read  for  particular  purposes.     The  second  objective 
Extends  Comprehension,  included  deliberate  kinds  of  analysis,  interpretation 
and  evaluation  that  a  student  might  use  when  participating  in  a  class 
discussion  or  that  a  reader  might  develop  for  a  talk  or  paper.     The  third 
objective,  Manages  the  Reading  Experience,  addressed  the  ability  of  readers 
to  adopt  different  strategies  depending  on  the  characteristics  of  different 
passages,  the  reader's  knowledge  and  experience  with  particular  kinds  of 
materials,  and  the  reader's  purpose  for  reading.     The  fourth  objective, 
Values  Reading,  addressed  how  readers  acquire  a  growing  appreciation  of  the 
ways  reading  can  affect  their  lives. 


2.7.2    Reading  Materials 

The  Learning  Area  Committee  suggested  that  a  variety  of  materials  were 
appropriate  for  use  in  the  assessment  and  that  they  be  "authentic",  i.e., 
reflect  the  realities  of  reading  passages,  charts,  and  instructions  found  in 
texts,  newspapers,  and  source  documents.     To  this  end,  the  committee 
suggested  that  material  appearing  in  the  reading  assessment  be  drawn  from 
many  sources,   including  stories,  science  and  social  studies  textbooks 
encyclopedias,  magazines,  news  articles,  peer  writing,  technical  writing,  and 
directions. 


2.7.3    Item  Development 

The  reading  xtems  newly  developed  for  the  1986  assessment  were  designed 
by  ETS  staff  and  by  outside  writers  trained  at  a  two-day  workshop  in 
Princeton.     The  new  items  were  reviewed  by  subject-matter  specialists  and 
editors  at  ETS  and  then  submitted  to  the  Learning  Area  Committee,  which 
conducted  a  final  review  and  selected  materials  for  field  testing  These 
materials  were  prepared  by  staff  and  submitted  to  0MB  clearance  prior  to 
field  testing.     After  field  testing  (described  in  section  2.13),   the  Learning 
Area  Committee  met  for  a  third  time  to  review  the  results  and  express  their 
preference  concerning  the  use  of  both  trend  and  newly  developed  items  in  the 
1986  assessment. 


Because  reading  is  not  generally  taught,  per  se,  after  the  elementary 
grade  levels,  the  decision  was  made  to  administer  the  same  assessment  at 
grade  7/age  13  and  grade  11/age  17  to  monitor  developmental  progress. 
(However,  subsequent  analyses  indicate  that  this  puts  an  undue  ceiling  on  the 
proficiency  levels  of  one  group  or  the  other;  in  1986,  a  more  challenging 
assessment  at  grade  11/age  17  would  have  been  preferable.)     The  items 
selected  by  the  Learning  Area  Committee  for  inclusion  in  the  1986  assessment 
underwent  final  review  by  subject-matter  specialists,  measurement  experts, 
and  editors,  as  well  as  a  review  to  detect  any  bias  according  to  the  ETS 
Standards  for  Quality  and  Fairness  (ETS,  1983). 

2.7.4    Reading  Background  and  Attitude  Questions 

In  developing  the  student  background  and  attitude  questions  particular 
to  reading,  the  Learning  Area  Committee  was  very  interested  in  addressing  not 
only  reading  instructional  techniques  but  also  some  aspects  of  the  reading 
objectives,  including  the  strategies  that  students  used  when  reading  and  the 
types  of  reading  they  were  likely  to  engage  in  both  during  and  outside  of 
school.     They  also  expressed  the  view  that  purpose  is  an  important  component 
of  reading,  but  might  be  difficult  to  tap  in  an  assessment  situation.  Along 
with  selecting  the  cognitive  materials  for  field  testing,  discussing 
background  and  attitude  questions  was  given  high  priority  at  the  second 
meeting  of  the  reading  Learning  Area  Committee  in  June  of  1984.  The 
background  and  attitude  items  were  field  tested  with  the  cognitive  items  and 
submitted  to  a  paral'^1  review  process.     Both  cognitive  and  noncognitive 
items  were  prepared  as  part  of  the  clearance  package  submitted  to  the 
government  for  review  and  0MB  clearance  for  the  1986  assessment. 

2.8    DEVELOPING  THE  MATHEMATICS  ASSESSMENT 
2.8.1  Objectives 

In  keeping  with  NAEP  procedures,  the  objectives  for  the  1986  mathematics 
assessment  (NAEP,  1986a)  were  derived  through  a  process  of  review  and 
revision.     First,  25  mathematics  educators  and  classroom  teachers  reviewed 
the  objectives  used  for  the  previous  assessment  in  1981-82.     Their  responses 
were  collated  by  staff  and  given  to  the  Learning  Area  Committee  at  its  first 
meeting  for  use  in  updating  the  objectives  for  the  1986  assessment.  The 
subsequent  draft  prepared  by  the  committee  was  reviewed  by  another  panel  of 
25  reviewers,  which  submitted  additional  comments  and  suggestions.  These 
responses,  collated  by  staff  members,  were  used  by  the  mathematics  Learning 
Area  Committee  to  prepare  the  final  set  of  mathematics  objectives. 

The  final  description  of  what  was  to  be  assessed  in  1986  was  organized 
according  to  a  matrix  of  five  broad  process  are?s  by  seven  content  areas. 
Although  all  objectives  were  intended  to  involve  elements  of  problem  solving, 
the  five  process  areas  included:  problem  solving/reasoning,  routine 
application,  understanding/comprehension,  skill,  and  knowledge.     The  content 
categories,  drawn  primarily  from  elementary  and  secondary  school  mathematics 
up  to  but  not  including  calculus,  included  the  following:  fundamental 
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methods  of  mathematics;  discrete  mathematics;  data  organization  and 
interpretation;  measurement;  geometry;  relations,  functions,  and  algebraic 
expressions;  and  numbers  and  operations. 

2.8.2  Item  Development 

Mathematics  items  were  newly  developed  for  the  1986  assessment  by 
external  item  writers,  staff,  and  the  Learning  Area  Committee  according  to 
detailed  specifications  set  forth  in  the  process  by  content  matrix.  Because 
of  the  increasing  availability  and  popularity  of  calculators,  NAEP  has 
gathered  information  about  their  use  by  students  beginning  with  the  1977-78 
assessment  and  this  practice  continued  in  1986.    A  minimal  amount  of 
instruction  on  the  use  of  the  calculator  is  given  prior  to  such  items.  Also, 
the  items  ara  repeated  without  the  use  of  calculators  in  other  parts  of  the 
assessment  to  permit  comparisons  of  performance  with  and  without  the 
calculator . 

The  process  and  schedule  for  developing  the  mathematics  items  were  very 
similar  to  those  used  for  reading.    At  the  January  1984  meeting,  the  Learning 
Area  Committee  issued  guidelines  for  designing  items,  which  were  carried  out 
by  staff  and  external  item  writers.     The  newly  developed  materials  were  then 
subjected  to  the  stringent  and  multiphase^  ETS  internal  review  process;  the 
results  were  reviewed  by  the  coamiittee  at  its  second  meeting  in  the  summer  of 
1984.     The  committee  made  revisions  and  wrote  new  items,  which  were 
subsequently  reviewed  and  prepared  for  0MB  clearance  prior  to  field  testing. 
After  field  testing,  the  results  were  reviewed  and  the  Learning  Area 
Committee  recommended  trend  and  newly  developed  items  to  be  included  in  the 
1986  assessment.     These  materials  were  again  subjected  to  the  ETS  and  NAEP 
review  process  and  eventually  submitted  for  OERI  review  and  0MB  clearance 
prior  to  being  included  in  the  1986  assessment. 

2.8.3  Background  and  Attitude  Questions 

In  developing  materials  for  the  student  questions  specific  to  its 
curriculum  area,  the  Learning  Area  Committee  considered  a  number  of  important 
contexts  for  learning  mathematics.     Five  categories  of  attitudinal  and 
background  measures  were  developed:     mathematics  in  school,  mathematics  and 
oneself,  mathematics  and  society,  mathematics  as  a  discipline,  and  attitudes 
toward  computers.     Additional  questions  were  included  to  gather  information 
on  each  student's  experience  with  calculators.     These  questions  covered  how 
often  the  student  had  used  a  calculator,  whether  the  student's  family  owned 
one,  in  what  courses  the  student  had  used  a  calculator,  and  what  experiences 
with  calculators  the  student  had  had  outside  of  school. 

2.9    DEVELOPING  THE  SCIENCE  ASSESSMENT 
2.9.1  Objectives 

The  framework  for  the  1986  science  assessment  was  developed  as  a  three- 
dimensional  matrix—content  by  context  by  cognition.     The  content  dimension 
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included  the  traditional  disciplines  of  science  (life  sciences,  physics, 
chemistry,  and  earth  and  space  sciences)  as  well  as  the  nature/processes  of 
science  and  its  history.     The  context  dimension  defined  four  types  of 
situations  for  presenting  assessment  items:     scientific,  personal,  societal, 
and  technological.     The  cognitive  dimension  identified  three  generic 
categories  required  to  deal  with  science  at  different  levels  of  complexity: 
knows,  uses,  and  integrates. 

The  process  for  developing  the  science  objectives  followed  the 
established  NAEP  pattern.     Before  the  first  meeting  of  the  Learning  Area 
Committee,  the  framework  used  for  the  1976-77  assessment  was  reviewed  by  25 
science  educators,  including  teachers,  administrators,  and  scientists.  At 
its  January  1984  meeting,   the  committee  discussed  these  reviews,  developed 
the  three-dimensional  matrix,  identified  the  major  categories  within  each 
dimension  for  assessment,  and  established  categories  for  student  self- 
reporting  of  attitude  and  background  information. 

The  first  draft  of  the  objectives  was  reviewed  by  15  practitioners, 
revised,  and  reviewed  again  by  about  25  individuals  representing  a 
combination  of  science  educators  and  interested  public  parties.  These 
reviews  were  collated  for  the  Learning  Area  Committee  and  used  to  create  the 
final  draft  of  the  objectives  booklet  (see  NAEP,  1986b). 

2.9.2  Item  Development 

More  than  400  new  science  items  were  written  by  about  35  external  item 
writers  according  to  specifications  prepared  by  NAEP  staff.     At  its  second 
meeting  in  June  1984,  the  Learning  Area  Committee  reviewed  all  items, 
designating  each  as  acceptable,  acceptable  with  revision,  or  rejected,  and 
checked  the  classification  of  each  item  according  to  the  matrix.  The 
remainder  of  the  process  used  to  develop  items  for  the  1986  science 
assessment  followed  that  described  earlier  for  reading  and  mathematics, 
consisting  of  reviews  for  content,  measurem':;nt ,  and  sensitivity  issues;  0MB 
clearance  for  field  testing;  field  testing  and  analysis  of  those  results;  a 
third  Learning  Area  Committee  meeting  for  final  selection  of  trend  and  newly 
developed  items;  ETS  and  NAEP  reviews  of  final  materials;  and  preparation  of 
materials  for  0MB  clearance  prior  to  their  inclusion  in  the  1986  assessment. 

2.9.3  Background  and  Attitude  Questions 

In  addition  to  drafting  the  three-dimensional  matrix  underlying  item 
development  for  the  1986  science  assessment,  the  Learning  Area  Committee 
spent  considerable  effort  formulating  areas  that  should  be  covered  in 
collecting  information  descriptive  of  students'  attitudes,  values,  and 
experiences  in  the  area  of  science.     Seven  categories  of  questions  for  self- 
reporting  by  students  were  defined:     attitudes  toward  science  classes,  career 
and  education  intentions,  socioscientif ic  responsibility,  science  as  a 
personal  tool,  value  of  science,  societal  issues,  and  experiences  in  science. 
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2.9.4    Developing  the  "Hands-on"  Pilot  in  Mathematics  and  Science 


In  conjunction  with  the  discussion  on  developing  materials  for  the  1986 
mathematics  and  science  assessments,  it  is  important  to  reference  NAEP's 
pilot  project  to  assess  higher-order  thinking  skills  in  science  and 
mathematics.     Well  documented  in  Learning  by  Doin^:     A  Manual  for  Teaching 
and  Assessing  Higher-Order  Thinking  in  Science  and  Mathematics  (NAEP,  1987b), 
as  well  as  in  the  final  project  report  entitled  A  Pilot  Study  of  Higher-Order 
Thinking  Skills  Assessment  Techniques  in  Science  and  Mathematics  (NAEP, 
1987c),  the  purpose  of  this  project  was  to  enhance  the  development  efforts  of 
the  mathematics  and  science  assessments.     As  stressed  in  the  introduction  to 
this  section  on  developing  the  1986  assessment  materials,  the  Assessment 
Policy  Committee  was  very  eager,  as  was  the  NAEP  staff,  to  measure  higher- 
order  thinking  skills.     However,  all  of  the  Learning  Area  Committees  felt  the 
frustration  of  trying  to  do  so  within  the  constraints  of  self -administered 
paper-and-pencil  tests. 

Thus,  the  staff  solicited  and  obtained  additional  funding  from  the 
National  Science  Foundation  to  prepare  and  conduct  a  pilot  of  assessment 
administrations  involving  "hands-on"  and  computer- administered  tasks  in 
science  and  mathematics.    A  very  exciting  and  well -received  project,  this 
work  will  be  continued  in  the  1990  science  and  mathematics  assessments. 


2.10    DEVELOPING  THE  COMPUTER  COMPETENCE  ASSESSMENT 
2.10.1  Objectives 

Developing  the  framework  for  the  1986  computer  competence  assessment 
presented  a  particular  challenge  to  NAEP  and  the  Learning  Area  Committee. 
Because  this  was  the  first  NAEP  assessment  in  this  subject  area,  the 
committee  had  to  develop  the  objectives  from  "scratch."    This  in  and  of 
itself  made  the  task  difficult;  however,  the  problem  was  exacerbated  by  the 
nature  of  instruction  and  learning  in  this  subject  area.     Because  of  the 
mixture  of  learning  about  computers  that  occurs  in  and  out  of  school, 
students  at  any  given  grade  level  can  have  a  wide  range  of  different' 
experiences,  and  students  at  higher  grade  levels  do  not  necessarily  have  more 
competence  than  students  at  lower  grade  levels.     Finally,  the  nature  of  the 
underlying  technology  makes  the  entire  field  "a  moving  target." 

Due  to  these  anticipated  difficulties,  the  computer  competence  Learniiig 
Area  Committee  was  composed  of  eight  members  as  compared  to  five  for  the 
other  subject  areas.     In  addition,  the  committee  had  many  more  meetings-- 
eight  in  all- -to  accomplish  its  complex  tasks  of  monitoring  development  of 
the  objectives  booklet  (NAEP,  1986c)  and  designing  as  well  as  reviewing 
items.     As  NAEP  procedures  dictate,  the  objectives  booklet  was  also  subjected 
to  a  wide-ranging  review  by  computer  specialists,  educators,  school 
administrators,  legislators,  members  of  the  business  community,  and  parents. 

The  final  objectives  were  arranged  in  a  framework  that  included  content 
and  cognitive  subdivisions.     The  content  domain  included  computer  knowledge 
(e.g.,  the  history  of  computing  and  uses  for  computer  technology),  computer 
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applications  (e.g.,  familiarity  with  the  operation  and  design  of  applications 
including  word  processing,  databases,  graphics,  spreadsheet,  and  lab 
instrumentation),  and  computer  science/programming  (e.g.,  knowledge  of 
programming  language  elements  and  structure  as  well  as  the  ability  to  plan 
and  design  programs).     The  cognitive  domain  included  knowledge,  applications, 
and  programming  categories,  and  certain  cognitive  skills  associated  with 
operation,  knowledge,  and  design. 

2.10.2  Item  Development 

For  the  computer  competence  assessment,  item  development  was  a 
particularly  challenging  and  frustrating  experience.     The  issue  of  "the 
moving  target"  described  above  was  everpresent.     No  sooner  were  prototype 
items  developed  than  advances  in  technology  made  them  obsolete.     However,  to 
be  clear  about  the  items  forming  the  basis  of  the  assessment,  the  Learning 
Area  Committee  included  examples  in  the  objectives  booklet.     A  second  and 
primary  concern  of  the  committee  centered  on  the  limitations  imposed  by 
trying  to  assess  students'  actual  ability  using  a  computer  with  only  paper - 
and-pencil  measures.     As  with  the  science  and  mathematics  assessments, 
additional  funds  for  a  "hands-on"  assessment  were  sought  from  the  National 
Science  Foundation.     However,   in  this  case- -and  in  retrospect,  probably 
appropriately  so- -a  pilot  assessment  of  this  nature  was  deemed  to  be 
premature . 

Eventually,  with  concerted  effort  by  staff,  the  Learning  Area  Committee, 
and  external  item  writers,  items  were  developed  for  all  aspects  of  the 
computer  competence  objectives.     After  lengthy  debate,  the  programming  items 
developed  for  grade  3/age  9  and  grade  7/age  13  used  BASIC  and  LOGO 
programming  languages,  whereas  those  developed  for  grade  11/age  17  used  BASIC 
and  Pascal  programming  languages. 

2.10.3  Background  and  Attitude  Questions 

The  student  background  questions  developed  for  the  computer  competence 
assessment  focused  on  two  very  important  issues- -attitudes  toward  computers 
and  access  to  learnJng  about  and  using  computers.     The  attitude  questions 
included  those  ou  students'  confidence  in  their  computing  ability;  thair 
feelings  on  the  value  of  knowing  about  computers  later  on  in  school  or  the 
workplace;  their  desire  for  more  challenging  computer  experiences;  and  their 
attitudes  toward  copyright  restrictions,  software  piracy,  and  computer 
ethics,     kccess  questions  included  those  on  the  availability  of  computers  at 
home  and  at  school;  the  various  opportunities  students  may  have  had  to  learn 
about  computers;  the  computer  courses  taken  by  students;  the  programming 
languages  students  know;  and  other  general  computer -related  information. 
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2.11    DEVELOPING  THE  U.S.  HISTORY  AND  LITERATURE  ASSESSMENTS 


2.11.1  Objec\:ives 

The  NAEP  consultants,  reviewers,  and  staff  involved  with  the  Foundations 
of  Literacy  project  (see  NAEP,  1986d)  agreed  that  a  full  assessment  of  U.S. 
history  not  only  would  include  recognition  of  the  facts  of  our  history- -the 
documents,  events,  and  personages  that  have  molded  the  nation- -but  also  would 
explore  the  extent  to  which  students  can  and  do  use  their  knowledge  to 
formulate  ideas  and  concepts,  to  recognize  patterns,  and  to  establish  for 
themselves  a  connectedness  of  things.     However,  having  knowledge  about  the 
v&riety  of  topics  that  form  the  basis  of  dialogue  and  information- sharing  is 
central  to  literacy,  and  the  U.S.  history  Learning  Area  Cojimittee  began  the 
process  of  describing  some  fundamentals.     Although  care  was  taken  to  address 
topics  relevant  to  political  history,  women's  history.  Black  history,  labor 
history,  technology,  geographies,  immigration,  and  foreign  policy,  the 
assessment  topics  were  arranged  chronologically  for  the  convenience  of 
ordering  the  material.     Seven  relatively  arbitrary  periods  of  history  were 
outlined:     Exploration  and  Colonization:     Up  to  1763;  the  Revolutionary  War 
and  the  New  Republic:     1763-1815;  Nationhood,  Sectionalism,  and  the  Civil 
War:     1815-1877;  Territorial  Expansion,  the  Rise  of  Modern  America,  and  World 
War  I:     1877-1920;  the  Great  Depression,  the  New  Deal,  and  World  War  II: 
1920-1945;  and  Post-World  War  II:  1945  to  Present. 

In  outlining  the  topics  for  the  literature  assessment,  the  Learning  Area 
Committee  felt  that  students  should  be  familiar  with  characters  who  have 
become  symbols  of  our  humanity;  with  authors  and  works  that  are 
representative  of  major  genres,  themes,  and  movements;  and  with  familiar 
quotations  from  poems,  plays,  speeches,  and  documents.     These  characters, 
authors,  works,  themes,  and  quotations  were  drawn  from  a  wide  variety  of 
literature,  including  classical  as  well  as  modern  works,  world  literature  in 
addition  to  American  ^nd  English  literature,  and  children's  classics.  The 
genres  included:     novels,  short  stories,  and  plays;  myths,  epics,  and 
biblical  stories;  poetry;  and  nonfiction. 

Developing  the  materials  for  the  Foundations  of  Literacy  project  started 
nearly  one  year  later  than  the  efforts  for  the  other  four  subject  areas 
comprising  the  1986  assessment.     However,  given  that  the  assessment  topics 
were  confined  to  the  knowledge  area  by  the  nature  of  the  grant  and  the 
assessments  were  only  given  to  one  grade/age  level  (17-year-olds  and  11th 
graders),  it  was  not  infeasible  to  work  within  such  a  compressed  schedule. 

2.11.2  Item  Develor-  ^nt 

A  10-member  Learning  Area  Committee  was  selected  by  NAEP  staff  and  by 
the  Foundations  of  Literacy  project  directors  from  the  Education  Excellence 
Network,  Chester  Finn  and  Diane  Ravitch.     Ccrprising  five  specialists  in  U.S. 
history  and  five  in  literature,  the  committee  met  three  times  during  the 
period  from  November  1984  through  June  1985.    Although  items  were  written  by 
NAEP  and  ETS  staff,  the  committee  reviewed  all  items  carefully  and  wrote  many 
new  ones.     They  reviewed  the  items  and  prepared  them  for  field  testing  at 
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their  second  meeting,  and  reviewed  the  results  of  the  field  test  at  their 
third  meeting. 

2.11.3    Background  and  Attitude  Questions 

The  U.S.  history  and  literature  Learning  Area  Committees  were  primarily 
concerned  with  gathering  information  about  students'  instructional 
experiences.     In  addition,  for  U.S.  history,  there  was  particular  concern 
that  high-school  history  courses  are  generally  taught  chronologically  and 
that  students  rarely  study  the  20th  century.     Thus,  questions  were  designed 
to  see  if  students  had  studied  the  time  periods  assessed.     For  literature, 
there  was  particular  concern  that  students  are  not  assigned  much  reading  in 
school  and  that  what  literature  is  assigned  is  quite  diverse  in  scope.  Thus, 
in  designing  the  background  questions  for  literature,  a  particular  attempt 
was  made  to  collect  information  about  students'  experience  with  basic 
literary  works  and  their  reading  habits. 

2.12    DEVELOPING  THE  COMMON  CORE  STUDENT  QUESTIONS  AND  QUESTIONNAIRES 

Five  additional  instruments  were  developed  for  the  1986  assessment:  a 
common  core  of  student  background  questions,  a  teacher  questionnaire,  a 
computer  coordinator  questionnaire,  a  school  characteristics  and  policies 
questionnaire,  and  an  excluded  student  questionnaire. 

The  student,  teacher,  and  school  instruments  were  designed  to  collect 
information  about  home,  classroom,  and  school  factors  related  to  eight  policy 
issues  that  were  the  focus  of  the  1986  assessment:  teacher  quality;  principal 
as  instructional  leader;  school  standards  and  policies;  school  environment 
conducive  for  education;  meeting  special  needs;  effective  classroom 
practices;  student's  school  experiences  and  attitudes;  and  home  environment 
supportive  of  education. 

The  development  of  policy  issues  and  items  was  an  iterative  process  that 
involved  staff  work,  field  testing,  periodic  review  by  an  external  advisory 
group,  and  review  by  the  Assessment  Policy  Committee.     An  initial,  longer 
list  of  policy  issues  was  developed  by  NAEP  staff  based  on  a  review  of 
current  policy  research  and  related  literature.     Of  particular  interest  were 
the  school  effectiveness  research,  the  push  for  raising  educational 
standards,  and  parents'   involvement  in  their  children's  education.  Items 
were  developed  to  assess  the  policy  issues  and  field  tested  with  students, 
teachers,  and  principals.     The  policy  issues,  items,  and  field  test  results 
were  reviewed  by  a  group  of  policy  researchers,  which  recommended  a 
consolidated  list  of  policy  issues  and  identified  specific  items  to  be 
included  in  the  final  questionnaires.     The  field  test  results  and  the 
recommendations  of  the  consultants  were  also  reviewed  by  the  Assessment 
Policy  Committee,     The  items  were  then  assembled  into  questionnaires  and 
submitted  to  internal  ETS  procedures  to  ensure  fairness  and  quality. 

Every  student  booklet  began  with  a  common  core  of  background  questions. 
In  many  cases  the  questions  used  in  1986  were  taken  from  prior  assessments. 
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Some  of  the  newly  developed  questions  focused  on  mathematics,   science,  and 
computer  instruction  and  others  on  parental  involvement  in  the  child's 
education.    Although  many  of  the  questions  were  common  to  the  three  grade/age 
levels  assessed,  some  were  specifically  targeted  to  elementary  or  high-school 
students.     At  grade  3/age  9,  the  background  questions  were  read  aloud  to  the 
students  and  took  approximately  15  minutes  to  complete.     At  the  other  two 
grade/age  levels,   students  read  and  answered  the  questions  on  t;ieir  own 
during  a  six-minute  time  period.     At  these  grade/age  levels,   th 2  questions 
asked  about  demographics,  home  environment,  and  instruction  experiences. 

The  teacher  questionnaire  was  given  to  a  sample  of  the  students' 
teachers,  and  took  approximately  20  minutes  to  complete.     The  major  part  of 
the  questionnaire  included  general  questions  about  demographics,  training, 
experience  and  classroom  management  strategies,  while  the  remaining  part 
included  questions  specific  to  reading/language  arts  and  English, 
mathematics,   science,  and,  at  grade  11/age  17,  U.S.  history.     At  grade  3/age 
9,  the  questionnaire  was  given  to  a  sample  of  the  students'  reading/language 
arts  teachers.     However,   it  was  assumed  that  in  most  cases  these  teachers 
also  taught  other  subject  areas,  so  the  questionnaire  included  general 
questions  plus  questions  about  reading,  mathematics  and  science  instruction. 
At  the  other  grade/age  levels,  participating  schools  were  randomly  designated 
as  reading,  mathematics,   science,  or  (at  grade  11/age  17  only)  U.S.  history 
schools,  and  questionnaires  were  given  to  a  sample  of  the  students'  teachers 
in  that  subject  area.     The  questionnaire  included  a  general  section  and 
several  sub j ect- specific  sections,  and  teachers  were  directed  to  fill  out  the 
appropriate  sections. 

The  computer  coordinator  questionnaire  was  administered  to  the  person  in 
each  partic5.pating  school  who  coordinated  computer  instruction  in  the  school. 
The  questionnaire  include  questions  about  the  coordinator's  background, 
training,  and  experience  and  about  the  types  of  computer  instruction  that 
were  present  in  the  school.     If  a  school  did  not  have  anyone  who  acted  in 
this  role,   the  questionnaire  was  not  administered. 

The  school  characteristics  and  policies  questionnaire  was  given  to  the 
principal  in  each  participating  school,  and  took  about  15  minutes  to 
complete.     The  questions  asked  about  the  principal's  background  and 
experience  and  about  school  policies,  programs,  facilities,  as  well  as  the 
composition  and  background  of  the  student  body. 

The  excluded  student  questionnaire  was  given  to  the  teachers  of  students 
who  were  identified  in  the  NAEP  sample  but  were  excluded  from  the  assessment 
for  some  reason,  usually  because  the  student  was  handicapped  or  had  limited 
English  proficiency.     This  questionnaire  took  approximately  three  minutes  per 
student  to  complete  and  asked  about  the  nature  of  the  student's  exclusion  and 
special  programs  in  which  the  student  participated. 


2.13    FIELD  TESTS  FOR  THE  1986  ASSESSMENT 

In  February  1985,   field  testing  commenced  for  the  1986  assessments  cf 
reading,  mathematics,   science,  computer  competence,  U.S.  history,  and 
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literature,  and  for  the  teacher  and  school  administrator  questionnaires 
described  previously. 

The  first  part  of  the  field  test  covered  reading,  mathematics,  science, 
and  computer  competence.     A  total  of  nine  booklets  was  developed  for  grade 
3/age  9,  each  containing  three  blocks  of  items.     In  all,  five  reading  blocks, 
eight  mathematics  blocks,  eight  science  blocks,  and  six  computer  competence 
blocks  were  field  tested  for  this  grade/age.    The  analysis  of  results  from 
the  grade  3/age  9  field  test  revealed  that  some  of  the  item  formats  were 
problematic  for  these  students;     therefore,  a  second  field  test  was  later 
conducted  at  this  grade/age  level  to  evaluate  revised  item  formats. 

For  grade  7/age  13,  12  booklets  were  developed  for  the  field  test, 
containing  three  blocks  each.    A  total  of  five  reading  blocks,  nine 
mathematics  blocks,  10  science  blocks,  and  12  computer  competence  blocks  was 
field  tested  for  this  grade/age.    Thirteen  booklets,  each  containing  three 
blocks,  were  created  for  the  grade  11/age  17  field  test;  these  booklets 
contained  a  total  of  five  reading  blocks,  10  mathematics  blocks,  12  science 
blocks,  and  12  computer  competence  blocks. 

Each  block  of  mathematics,  science,  reading,  and  computer  competence 
items  contained  two  minutes  of  subject-specific  background/attitude  items  and 
14  minutes  of  cognitive  items,  as  well  as  two  minutes  of  general  background 
items.     Thus,  each  booklet  contained  a  total  of  approximately  48  minutes  of 
assessment  material. 

The  ;iecond  part  of  the  field  test  covered  U.S.  history  and  literature 
items  for  grade  11/age  17  only.     Three  booklets  were  created  for  each  of 
these  subjects,  each  containing  three  blocks.     Each  booklet  contained 
approximately  six  minutes  of  background  items  and  48  minutes  of  cognitive 
items . 

From  February  through  May  1985,  trained  NAEP  and  ETS  staff  conducted 
field  tests  for  the  reading,  mathematics,  science,  computer  competence,  U.S. 
history,  and  literature  items  in  285  classrooms,  representing  an  estimated 
7,550  students  from  61  school  districts.     In  addition,  94  school  officials 
and  272  classroom  teachers  were  asked  to  complete  the  school  and  teacher 
questionnaires.     The  set  of  schools  represents  a  purposive  national  sample, 
consisting  of  medium- sized  cities,  small  places,  and  disadvantaged  urban 
areas  of  large  cities  in  the  country's  four  geographical  regions.     Field  test 
sites  were  selected  and  solicited  for  participation  by  NAEP  staff.  Within 
each  selected  site,  school  personnel  identified  the  3rd-,  7th-,  and  11th- 
grade  students  to  be  surveyed. 

The  field  test  data  were  scored  and  analyzed  from  March  to  May  1985. 
Using  interleaved  item  analysis,  which  provides  the  mean  percentage  of 
correct  responses  for  each  item  in  the  field  test,  NAEP  staff  and  consultants 
reviewed  the  mat^.rials  according  to  five  purposes:     to  determine  which  items 
were  most  related  to  achievement  in  the  four  subject  areas;  to  evaluate  the 
effectiveness  of  items  designed  specifically  to  assess  higher- order  thinking 
skills;  to  determine  necessary  revisions  to  items  that  lacked  clarity,  or  to 
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ineffective  item  formats;  to  prioritize  items  to  be  included  in  the  full 
assessment;  and  to  determine  appropriate  timing  for  assessment  items. 

Reviews  of  field  test  items  were  conducted  by  NAEP  and  ETS  staff  the 
Learning  Area  Committee,  and  external  consultants;  through  this  process 
items  were  selected  for  the  1986  assessment.    A  final  clearance  package' 
containing  these  items  was  submitted  to  0MB  in  July  1985. 

2.14    FINAL  PREPARATION  OF  THE  1986  ASSESSMENT  MATERIALS 
2.14.1    Objectives  Booklets 

After  consensus  review  and  approval  by  each  Learning  Area  Committee  the 
objective  booklets  were  sent  to  OERI  for  the  internal  government  and  peer 
review  process.     Subsequent  to  that  30-day  review  and  incorporation  of 
suggested  revisions,  the  booklets  were  submitted  to  ETS  internal  editing  and 
finally  to  the  composition  and  printing  process.     The  four  objectives 
booklets  (reading,  mathematics,  science,  and  computer  competence)  as  well  as 
the  description  booklet  for  the  Foundations  of  Literacy  project- -the  U  S 
history  and  literature  assessments- -were  published  and  released  in  1986 


2.14.2    Student  Assessment  Booklets  and  Questionnaires 

The  items  earmarked  by  each  Learning  Area  Conunittee  and  submitted  to  0MB 
clearance  for  inclusion  in  the  1986  assessment  were  assembled  into  blocks  in 
accordance  with  the  assessment  design.     These  blocks  were  assembled  to  meet 
committee  content,  context,  and  cognition  specifications  and  to  conform  to 
the  assessment  time  and  administration  restrictions.    Approximately  two 
minutes  of  subject-matter  relevant  background  questions  appeared  at  the  start 
of  most  blocks,  followed  by  the  cognitive  items.     Specifically,  for  reading 
SIX  Item  blocks  were  prepared  for  assessment  at  each  grade/age  level-  for 
mathematics  and  science,  seven  item  blocks  were  prepared  for  grade  S/age  9 
nine  for  grade  7/age  13,  and  11  for  grade  11/age  17;  and  for  computer  ' 
competence,   three  item  blocks  were  prepared  at  grade  3/age  9  and  six  item 
blocks  were  prepared  for  each  of  the  two  older  grade/age  levels.  Similarly 
the  questions  for  school  administrators,  teachers,  and  about  excluded 
students  were  assembled  into  questionnaires. 

The  assessment  blocks  and  questionnaires  were  subjected  to  the  review 
process  established  by  ETS  and  NAEP,  including  scrutiny  by  subject-matter 
specialists,  measurement  specialists,  test  editors,  and  persons  specially 
trained  to  review  questions  for  any  potential  sensitivity  concerning  women  or 
minority  groups.     As  part  of  the  0MB  clearance  process,  all  items  were  also 
reviewed  by  OERI  staff,  by  the  Office  for  Quality  Assurance  and  by  the  Office 
for  Management  and  Budget.     Subsequent  to  the  complete  review  process  the 
blocks  and  questionnaires  were  submitted  for  composition,  printing  bundling 
and  distribution  to  the  Westat,  Inc.,  field  staff  responsible  for 
administering  the  1986  asses^mont. 


2.15     1986  ASSESSMENT  DEVELOPMENT  CONSULTANTS 
Reading  Learning  Area  Committee 

Naomi  Gordon,  Public  Schools  of  Brookline,  Lexington,  MA 
Judith  Langer,  University  of  California,  Berkeley,  CA 
Dorothy  Strickland,  Columbia  University,  New  York,  NY 
Robert  Tierney,  University  of  Illinois,  Champagne -Urbana,  IL 
Richard  Venezky,  University  of  Delaware,  Newark,  DE 


Reading  Item  Development  Consultants 

Jo  Beth  Allen,  Kansas  State  University,  Manhattan,  KS 

Arthur  Applebee,  National  Council  of  Teachers  of  English,  Urbana,  IL 

Michael  Axline,  University  of  Oregon,  Eugene,  OR 

Fernie  Baca,  University  of  Colorado,  Denver,  CO 

Richard  Beach,  University  of  Minnesota,  Minneapolis,  MN 

Barbara  Bianchi,  Paideia  School,  Atlanta,  GA 

Susan  Blank,  Manpower  Demonstration  Research  Corporation,  Brooklyn,  NY 
Robin  Butterfield,  Northwest  Regional  Educational  Laboratory,  Portland,  OR 
Robert  Calfee,  Stanford  University,  Stanford,  CA 

Jeanne  Chall,  Harvard  Graduate  School  of  Education,  Cambridge,  MA 
Carita  Chapman,  Chicago  Public  Schools,  Chicago,  IL 

Ruth  Coleman,  North  Side  High  School  Mothers  Alumni  Club,  Fort  Wayne,  IN 

Christopher  Connell,  Associated  Press,  Washington,  DC 

Larry  Coon,  McDonald's  Restaurants,  Henderson,  TX 

Bernice  Cullinan,  New  York  University,  New  York,  NY 

Mary  E.  Curtis,  Harvard  Graduate  School  of  Education,  Cambridge,  MA 

Jacqueline  Danzberger,  Youthwork  Inc.,  Arlington,  VA 

Martha  Darling,  Washington  Roundtable  Education  Study,  Bellevue,  WA 

Philip  DiStefano,  University  of  Colorado,  Boulder,  CO 

Terry  Dozier,  South  Carolina  State  Department  of  Education,  Columbia,  SC 
Priscilla  Drum,  University  of  California  at  Santa  Barbara,  Santa  Barbara,  CA 
William  Eller,  State  University  of  New  York  at  Buffalo,  Amherst,  NY 
Leo  Estrada,  University  of  California,  Los  Angeles,  CA 
Claryce  Evans,  Boston  Public  Schools,  Boston,  MA 

Marjorie  Farmer,  School  District  of  Philadelphia,  Philadelphia,  PA 

Roger  Farr,  University  of  Indiana,  Bloomington,  IN 

Edmund  Farrell,  University  of  Texas,  Austin,  TX 

Edward  Fry,  Rutgers  University,  New  Brunswick,  NJ 

Carol  Gibson,  National  Urban  League,  New  York,  NY 

Kenneth  Goodman,  University  of  Arizona,  Tucson,  AZ 

Donald  Graves,  University  of  New  Hampshire,  Durham,  NH 

Jean  Greenlaw,  North  Texas  State  University,  Denton,  TX 

Doris  Hankins,  Germantown  High  School,  Germantown,  TN 

Jerome  Harste,  University  of  Indiana,  Bloomington,  IN 

David  Hayes,  University  of  Georgia,  Athens,  GA 

Paul  Heffernan,  Star  Market,  Newtonville,  MA 

Harold  Herber,  Syracuse  University,  Syracuse,  NY 

Jane  Holt,  Champlain  Valley  Union  High  School,  Hinesburg,  VT 
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Shu-in  Huang,  City  of  Thornton,  Thornton,  CO 

Evaline  Khayat  Kruse,  Audubon  Junior  High  School,  Los  Angeles,  CA 
Diane  Lapp,  Boston  University,  Boston,  MA 

Herbert  J.  Lapp,  Jr.,  GPU  Nuclear  Corporation,  Parsippany,  NJ 
Ron  Lessnau,  Hamburger  University,  Oakbrook,  IL 
Ray  Marshall,  University  of  Texas,  Austin  TX 

Phyllis  A.  Miller,  Reading  Development  Seminars,  Minneapolis  MN 
Charles  Moody,  University  of  Michigan,  National  Alliance  of  Black  School 
Educators,  Ann  Arbor,  MI 

Peter  Mosenthal,  Syracuse  University,  Syracuse,  NY 

Edwin  Newman,  NBC  News,  New  York,  NY 

Pedro  Pedraza  Jr.,  Hunter  College,  New  York,  NY 

Anthony  Petrosky,  University  of  Pittsburgh,  Pittsburgh,  PA 

Carolyn  N.  Pinckney,  Bunker  Hill  Elementary  School,  Washington,  DC 

Carolyn  Pollan,  State  of  Arkansas,  Fort  Smith,  AR 

Walter  L.  Powers,  School  District  #271,  Coeur  d'Alene,  ID 

John  Readance,  Louisiana  State  University,  Baton  Rouge,  LA 

Beverly  Roller,  Jefferson  County  Public  Schools,  Littleton,  CO 

Glenn  E.  Rotz ,  Highland  Elementary  School,  Clarkson,  WA 

Sarah  Saint-Onge,  Godine  Publishing  Co.,  Boston,  MA 

Adan  C.  Salgado,  Johnston  High  School,  Austin,  TX 

S.  Jay  Samuels,  University  of  Minnesota,  Minneapolis,  MN 

Robert  Schreiner,  University  of  Minnesota,  Minneapolis,  MN 

John  Stewig,  University  of  Wisconsin,  Milwaukee,  WI 

Jaap  Tuinman,  Simon  Fraser  University,  Burnaby,  B.C.,  Canada 

Janet  Tully,  Marriott  Corporation,  Washington,  DC 

Richard  Vacca,  Kent  State  University,  Kent,  OH 

Rod  Vahl,  Central  High  School,  Davenport,  lA 

Sheila  Valencia,  University  of  Colorado,  Boulder  CO 

Thomas  Vallejos,  University  of  Colorado,  Boulder,  CO 

Maria  Watkins,  University  of  Pennsylvania,  Graduate  School  of  Education 

Philadelphia,  PA 
Rick  Wetherell,  North  Bend  High  School,  West  Bend,  OR 
Susan  M.  Wolf,  The  Hastings  Center,  Hastings -on -Hudson,  NY 
Kathy  Yen,  San  Francisco  Public  Schools,  San  Franciso,  CA 
Seymour  Yesner,  Brookline  High  School.  Brookline,  MA 


Mathematics  Learning  Area  Committee 

James  Bruni,  Herbert  H.  Lehman  College.  Bronx,  NY 

Iris  M.  Carl,  Houston  Independent  School  District,  Houston,  TX 

Clyde  L.  Corcoran,  California  High  School,  Whittier,  CA 

F.  Joe  Crosswhite,  The  Ohio  State  University,  Columbus,  OH 

Shirley  Hill,  University  of  Missouri -Kansas  City,  Kansas  City,  MO 


Mathematics  Consultants 

Joan  L.  Akers,  San  Diego  County  Office  of  Education,  San  Diego,  CA 
Rudy  B.  Beede,  Forrest  City  Middle  School,  Forrest  City,  AR 
Elaine  Bologna,  Summit  School,  Winston-Salem,  NC 


-55- 


67. 


Olyrapia  E.  Boucree,  New  Orleans  Public  Schools,  New  Orleans,  LA 
Donna  Kay  Buck,  Washington-Hoyt  Schools,  Tacoma,  WA 

Ernestine  Capehart,  West  Virginia  Department  of  Education,  Charleston,  WV 
Thomas  P.  Carpenter,  University  of  Wisconsin.  Madison,  WI 
Mary  Kay  Corbitt,  Louisiana  State  University,  Baton  Rouge,  LA 
Gilbert  J.  Cuevas,  University  of  Miami,  Miami,  FL 

Edgar  L.  Edwards,  Jr.,  Virginia  State  Department  of  Education,  Richmond,  VA 
Carolyn  K.  Ehr,  Fort  Hays  State  University,  Hays,  KS 
James  Fey,  University  of  Maryland,  College  Park,  MD 
Carole  Greenes,  Boston  University,  Boston,  MA 

Marilyti  L.  Hala,  South  Dakota  Department  of  Education,  Pierre,  SD 

Jerry  Jean  Hale,  Southmoor  Elementary  School,  Denver,  CO 

Norma  G.  Hernandez,  University  o"^  Texas,  El  Paso,  Tx 

Patricia  M.  Hess,  Albuquerque  Public  Schools,  Albuquerque,  NM 

Vernon  R.  Hood,  Portland  Community  College,  Portland,  OR 

Larry  Johnson,  Clarke  County  Schools,  Athens,  GA 

Mary  Grace  Kantowski,  University  of  Florida,  Gainesville,  FL 

Robert  Kenney,  Vermont  Department  of  Education,  Montpelier,  VT 

Genevieve  M.  Knight,  Coppin  State  College,  Baltimore,  MD 

Katherine  P.  Layton,  Beverly  Hills  High  School,  Beverly  Hills,  CA 

John  LeBlanc,  University  of  Indiana,  Blooraington,  IN 

Gordon  Lewis,  District  of  Columbia  Public  Schools,  Washington,  DC 

Betty  K.  Lichtenberg,  University  of  South  Florida,  Tampa,  FL 

Mary  M.  Lindquist,  Columbus  College,  Columbus,  GA 

Mary  Martin,  Lawrence  School,  Brookline,  MA 

William  D.  McKillip,  University  of  Georgia,  Athens,  GA 

James  A.  Nelson,  Simmons  Junior  High  School,  Aberdeen,  SD 

Phares  G.  O'Daffer,  Illinois  State  University,  Normal,  IL 

Joseph  N.  Payne,  University  of  Michigan,  Ann  Arbor,  MI 

Norma  J.  Reed,  Douglas  Smith  Elementary  School,  Alief,  TX 

Robert  Reys ,  University  of  Missouri,  Colvimbia,  MO 

Joyce  Scalzitti,  Summit  Junior  High  School,  Paterson,  NJ 

Bernie  Richard  Schnorenberg,  Sundance  Junior-Senior  High  School,  Sundance,  WV 

Helene  Silverman,  Herbert  H.  Lehman  College,  Bronx,  NY 

Louise  M.  Smith,  Charleston  County  Schools,  Charleston,  SC 

William  A.  Stannard,  Eastern  Montana  College,  Billings,  MT 

Jacquelyn  M.  Stonebraker,  Evergreen  Junior  High  School,  Salt  Lake  City,  UT 

Marilyn  N.  Suydam,  The  Ohio  State  University,  Columbus,  OH 

Jane  Swafford,  Northern  Michigan  University,  Marquette,  MI 

B.  Ross  Taylor,  Minneapolis  Public  Schools,  Minneapolis,  MN 

Alexander  Tobin,  The  School  District  of  Philadelphia,  Philadelphia,  PA 

Kenneth  J-  Travers ,  University  of  Illinois,  Champaign,  IL 

Gloria  H.  Tuggle,  Memphis  City  Schools,  Memphis,  TN 

Charlene  E.  West,  Cajon  Valley  School  District,  El  Cajon,  CA 

Elizabeth  M.  Whitaker,  Lee  Resource  Center,  Tucson,  AZ 

Steven  S.  Willoughby,  New  York  University,  New  York,  NY 

James  W.  Wilson,  University  of  Georgia,  Athens,  GA 

Marcelline  Richard  Zambuco,  John  F.  Deering  Junior  High  School,  Warwick,  RI 
Marilyn  Zweng,  University  of  Iowa,  Iowa  City,  lA 
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Audrey  Champagne,  American  Association  for  the  Advancement  of  Sci 

Washington,  DC 
George  Kieffer,  University  of  Illinois,  Urbana,  IL 
Nuria  Rodriguez,  Beverly  Hills  High  School,  Beverly  Hills,  CA 
Calvin  VanderWerf,  University  of  Florida,  Gainesville,  Fl' 
Wayne  Welch,  University  of  Minnesota,  Minneapolis,  Mn' 


Science  Consultants 

Don  Ahshapanek,  Haskell  Indian  Junior  College,  Lawrence,  KS 
Dale  R.  Baker,  University  of  Utah,  Salt  Lake  City,  UT 
Paul  Bamberg,  Harvard  University,  Cambridge,  MA 
Gail  Barcelo,  Board  of  Education,  South  Brunswick,  NJ 

Ron  Barnes,  Office  of  Science,  Baltimore  County  School  District,  Baltimore, 
MD  ' 

William  E.  Barstow,  University  of  Georgia,  Athens,  GA 
Linda  Behrens,  Perry  Hall  Middle  School,  Baltimore,  MD 
Jack  Bouma,  Pikesville  Middle  School,  Baltimore,  MD 
Marilyn  Braveman,  American  Jewish  Committee,  New  York,  NY 
Linda  Broach,  Mountainbrook  Jr.  High  School,  Birmingham,  AL 
John  R.  Brown,  Jr.,  Trustee,  Kean  College  of  NJ,  Short  Hills,  NJ 
William  S.  Dougall,  Lakeside  School,  Seattle,  WA 

Kenneth  W.  Dowling,  Wisocnsin  Department  of  Public  Instruction,  Madison  WI 
Beulah  Durr,  Nottingham  High  School,  Syracuse,  NY 
Vera  King  Farris,  Richard  Stockton  State  College,  Pomona,  NJ 
Louis  Finsand,  Malcolm  Price  Laboratory  School,  University  of  Northern  Iowa 
Cedar  Falls,  lA 

Joseph  M.  Flynn,  Citizen's  Council  for  Ohio  Schools,  Cleveland  OH 

Anthony  Galitsis,  N.Y.C.  Board  of  Education,  Brooklyn,  NY 

Roy  A.  Gallant,  University  of  Southern  Maine,  Portland,  ME 

Beverly  German,  Shady  Spring  Elementary  School,  Baltimore,  MD 

Reen  Gibb,  Brookline  High  School,  Brookline,  MA 

Dennis  Gray,  Council  for  Basic  Education,  Washington,  DC 

Martha  Green,  Florida  Department  of  Education,  Tallahassee,  FL 

Laura  Hallowell,  Wellesley  High  School,  Wellesley,  MA 

Tohn  Harkness,  Wausau  West  High  School,  Wausau,  WI 

Charles  J.  Hemenway,  St.  Paul's  Episcopal  School,  New  Orleans,  LA 

Dale  T.  Hesser,  North  Syracuse  School  District,  North  Syracuse,  NY 

Vincent  Hollis,  Howard  University,  Washington,  DC 

Ann  C.  Howe,  North  Carolina  State  University, ^Raleigh,  NC 

Cindy  Hrebar,  Jefferson  County  School  District,  Lakewood,  CO 

Ken  Indek,  University  of  Illinois,  Champaign,  IL 

Bernard  H.  Johnson,  Central  State  University,  Wilberforce,  OH 

Russ  Jones,  Sussex  Elementary  School,  Balt^'  ^ore,  MD 

Maryanne  Kalin,  Francis  W.  Parker  School,  Chicago,  IL 

James  Kelly,  Malcolm  Price  Laboratory  School,  University  of  Northern  Iowa 

Cedar  Falls,  lA 
Leopold  E.  Klopfer,  University  of  Pittsburgh,  Pittsburgh,  PA 
Sylvia  Kr-^mer,  Women's  Action  Alliance,  New  York,  NY 
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Karl  Kranz,  National  Zoo,  Washington,  DC 

Charles  Kuzrainski,  Hopewell  Valley  Regional  School  District,  Pennington,  NJ 

Terry  Kwan,  Houghton-Mifflin  Reference  Division,  Boston,  MA 

Jeanne  Liu,  South  Carolina  Department  of  Education,  Colxambia,  SC 

Steven  Maltese,  Charlesmont  Elementary  School,  Baltimore,  MD 

Phyllis  Marcuccio,  National  Science  Teachers  Association,  Washington,  DC 

George  F.  Matthias,  Cro ton-Harmon  Senior  High  School,  Croton-on-Hudson,  NY 

Alan  J,  McCormick,  Laramie,  WY 

John  McGeehee,  Rolling  Hills  High  School,  Rolling  Hills  Estates,  CA 
Thomas  McGuire,  Briarcliff  High  School,  Briarcliff  Manor,  NY 
Robert  McNeesh,  Baltimore  County  Public  Schools,  Towson,  MD 
Kenneth  Mechling,  Clarion  University,  Clarion,  PA 

James  Mecklenburger ,  National  School  Boards  Association,  Alexandria,  VA 

Ida  M.  Medlem,  Bellaire  High  School,  Bellaire,  TX 

Suzanne  Mittenthal,  University  of  Illinois,  Urbana,  IL 

Pat  Morris,  University  of  Illinois,  Champaign,  IL 

Patsy  Mueller,  Highland  Park  High  School,  Highland  Park,  IL 

John  Murphy,  Board  of  Education,  Howard  County  Public  Schools  System,  MD 

Joseph  Novak,  Cornell  University,  Ithaca,  NY 

Linda  Parker,  Department  of  Education,  Tallahassee,  FL 

George  Patrinicola,  Loch  Raven  Senior  High  School,  Baltimore,  MD 

Nadine  Peterson,  Oak  Park  and  River  Forest  High  School,  Oak  Park,  IL 

Michael  D.  Piburn,  Salt  Lake  City,  UT 

Frank  Quagliano,  Macintosh  Middle  School,  Sarasota,  FL 

Senta  Raizen,  National  Academy  of  Science,  Washington,  DC 

Thomas  R,  Richardson,  University  School  of  Milwaukee,  Milwaukee,  WI 

Joseph  Riley,  Michigan  Department  of  Education,  Lansing,  MI 

David  Robinson,  Albany  State  College,  Albany,  GA 

James  T.  Robinson,  Boulder  Valley  Schools,  Boulder,  CO 

Eloy  Rodriguez,  University  of  California,  Irvine,  CA 
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Chapter  3 
SAMPLE  DESIGN 


Morris  H.  Hansen,  Keith  Rust,  and  John  Burke 
West at,  Inc. 


The  sample  of  students  for  the  1986  NAEP  was  selected  using  a  complex 
multistage  sample  design  involving  the  sampling  of  students  from  selected 
schools  within  94  selected  geographic  areas,  called  primary  sampling  units 
(PSUs),  from  across  the  United  States.     The  sample  design  is  described  in 
detail  in  National  Assessment  of  Kdnrational  Progress- -17th  Year  Sampling  and 

Weip;hting  Procedures.  Final  Report  (Burke,  Braden,  Hansen,  Lago,  6c  Tepping, 

1987).     This  chapter  provides  an  overview  of  the  design. 

Although  generally  similar  in  nature  to  the  sampling  procedures  used  in 
the  1984  assessment,  a  number  of  new  features  were  introduced  in  the  1986 
design.     The  most  important  of  these  are  listed  below. 

1)  The  definition  of  primary  sampling  units  and  their 
stratification  was  changed.     In  particular,  whole 
metropolitan  statistical  areas  were  generally  defined  as 
PSUs.     The  34  largest  of  these  were  sampled  with 
certainty. 

2)  The  number  of  PSUs  sampled  was  increased  from  64  to  94. 

3)  "Bridge"  samples  were  defined  for  the  assessment  of 
13-year-olds  in  fall  and  9-year-olds  in  winter,  using  a 
subsample  of  64  of  the  94  selected  PSUs.     The  bridge 
samples  were  used  for  certain  of  the  bridging  studies, 
needed  for  comparisons  with  earlier  NAEP  assessments, 
which  took  place  in  fall  for  13 -year-olds  and  winter  for 
9-year-olds . 

4)  In  schools  with  enrollment  of  small  to  moderate  size,  all 
eligible  students  were  selected  for  assessment  in  sampled 
schools.     In  larger  schools,  a  sample  of  eligible 
students  was  assessed.     Overall  there  was  a  moderate 
increase  in  the  numbers  of  students  assessed  per  school, 
compared  with  1984. 

5)  Efforts  were  made,  through  varying  the  probabilities  of 
selection  of  PSUs  and  schools,  to  increase  the  proportion 
of  the  overall  sample  comprising  Black  and  Hispanic 
students  for  each  age  class. 
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In  the  first  stage  of  sampling,  the  United  States  (the  50  states  and  the 
District  of  Columbia)  was  divided  into  geographic  primary  sampling  units 
(PSUs),  each  comprised  of  either  a  metropolitan  statistical  area  (MSA),  a 
single  county,  or  a  group  of  contiguous  counties.     Each  PSU  met  a  minimum 
size  requirement,  based  on  1980  Census  population  data.     The  use  of  whole 
MSAs  as  PSUs  was  a  departure  from  the  procedure  used  in  the  1984  and  previous 
years.     This  change  was  designed  to  reduce  sampling  variance,  particularly 
for  estimates  relating  to  minority  groups.     With  the  use  of  whole  MSAs  as 
PSUs,  the  large  PSUs  are  considerably  larger  and  more  heterogeneous  than  with 
the  use  of  counties  as  first-stage  units.     One  consequence  is  a  smaller 
variance  between  PSUs  within  noncertainty  strata.     Another  consequence  is 
that  the  PSUs  that  come  into  the  sample  with  certainty  account  for  a  high 
proportion  of  the  population  (roughly  one  third  of  the  total  population  and 
approximately  one  half  of  the  Hispanic  and  the  Black  populations  are  included 
in  PSUs  selected  with  certainty) ,  whereas  with  the  larger  PSUs  defined  in 
terms  of  whole  counties,  as  in  the  past,  certainty  selections  accounted  for 
roughly  5  percent  of  the  population.     These  two  factors  combined  greatly 
reduce  the  b^>.tween-PSU  contribution  to  variance,  with  little  or  no  added  cost 
for  travel  within  PSUs. 

Twelve  subuniverses  of  PSUs  were  defined.     The  PSUs  were  classified  into 
four  ragions,  each  containing  about  one  quarter  of  the  U.S.  population.  In 
each  region,  PSUs  were  classified  as  MSA  or  non-MSA.     In  the  Southeast  and 
West  regions,  the  PSUs  were  further  classified  as  high  minority  (20  percent 
of  the  population  in  the  1980  Census  was  either  Black  or  Hispanic)  or  not. 
The  resulting  subuniverses  are  shewn  in  Table  3.1.     Among  the  larger  PSUs,  34 
were  large  enough  to  be  designated  as  certainty  units  and  were  selected  with 
probability  one.     Within  each  major  stratum  (the  subuniverses),  further 
stratification  was  achieved  by  ordering  the  noncertainty  PSUs  according  to 
several  additional  socioeconomic  characteristics,  yielding  60  strata.  One 
PSU  was  selected  with  probability  proportional  to  size  from  each  of  the  60 
noncertainty  strata.     PSUs  within  the  high-minority  subuniverses  were  sampled 
at  twice  the  rate  of  PSUs  in  the  other  subuniverses. 

The  increase  in  sample  size  from  64  to  94  PSUs  was  also  aimed  at 
decreasing  the  component  of  variance  contributed  by  sampling  at  the  first 
stage,  and  thus  reducing  the  variances  of  sample  estimates.     This  gain  was 
rendered  cost-effective  in  part  by  the  fact  that  all  of  the  main  NAEP 
assessments  were  conducted  simultaneously  in  the  spring,  rather  than  at  a 
separate  time  for  each  grade/age,  so  that  there  was  sufficient  work  within 
each  PSU  to  keep  one  or  more  teams,  consisting  of  a  supervisor  and  exercise 
administrators,   fully  occupied  throughout  the  assessment  period.  Increasing 
the  number  of  PSUs  also  gave  rise  to  somewhat  of  an  increase  in  the  precision 
of  sampling  error  estimates  obtained  using  the  jackknife  approach,  discussed 
in  Chapter  14.     As  noted  below,  a  subsample  of  64  of  the  94  PSUs  was  used  for 
the  smaller  bridge  samples,  conducted  in  the  fall  and  winter,  to  maintain 
their  cost  efficiency. 
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Table  3.1 
The  Sampling  Subuniverses 
and  the  Number  of  Noncertainty  Strata  in  Each 


MSA  PSUs 


Region 

Regular 

High  minority 

Strata 

Strata 

Northeast 

8 

Southeast 

4 

6 

Central 

8 

West 

4 

6 

Total 

24 

12 

Non-MSA  PSUs 
Regular         High  minority 

Strata  Strata 


2 
4 
6 
4 

16 


6 
2 
8 


These  PSUs  were  used  for  both  the  spiral  assessments  and  the  bridge 
assessments  conducted  in  the  spring.     The  bridge  assessments  conducted  in  the 
fall  and  winter  used  a  subsample  of  64  PSUs  which  were  selected  from  the 
complete  set  of  94  PSUs.     The  reduced  number  of  PSUs  used  in  the  fall  and 
winter  arose  because  the  sample  sizes  of  students  at  each  age  were  much 
smaller  than  for  the  spring  assessment,  so  that  cost  considerations  dictated 
that  the  number  of  PSUs  selected  should  be  reduced  for  these  samples  In 
subsampling  the  PSUs.   the  18  largest  certainty  PSUs  were  retained,  while  46 
of  the  76  remaining  PSUs  were  retained  with  probability  proportional  to  a 
measure  of  size  of  the  stratum  from  which  they  were  drawn. 

Further  details  of  the  stratification  and  sampling  of  PSUs  are  eiven  in 
Burke  et  al.   (1987).  sections  2.1-2.3.  g      n  in 


3.2    SAMPLING  OF  SCHOOLS 

In  the  second  stage  of  sampling,  the  public.  Catholic,  other  private 
Bureau  of  Indian  Affairs,  and  Department  of  Defense  schools  within  each  of 
the  94  PSUs  were  listed  according  to  the  three  grade/age  groups.  An 
independent  sample  of  schools  was  selected  separately  for  each  of  the 
grade/age  groups.     Thus,  some  schools  were  selected  for  assessment  of  two 
grade/age  groups,  and  a  few  for  all  three  groups. 

The  list  of  schools  was  derived  from  the  fall  1984  list  of  U.S. 
elementary  and  secondary  schools  provided  by  Quality  Education  Dataj  Inc. 
This  frame  included  information  on  school  enrollment  and  grade  span!  as  well 
as  unique  identification  and  address  information. 

Five  subframes  of  schools  were  formed  from  the  complete  list  of  schools 
within  the  94  PSUs.     One  frame  was  created  for  each  grade/age  group  for  the 
main  NAEP  assessment,  with  an  additional  frame  each  for  both  the  age  9  and 
age  13  Bridge  A  samples.     Each  subframe  was  designed  to  include  schools  that 
in  the  aggregate  were  estimated  to  contain  all  but  a  trivial  number  of 
students  in  the  eligible  age  range  for  the  sample  in  question.     In  each  case 
the  frame  coverage  of  the  relevant  age  was  estimated  to  be  in  excess  of  99 
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percent.     Table  3.2  below  shows  the  grade  span  definitions  used  for  the 
different  frames.    Any  school  having  any  one  or  more  of  the  relevant  grades 
was  included  in  the  appropriate  frame.     Thus,  for  example,  a  school  with 
grades  1  through  6  was  included  in  both  of  the  age  9  frames,  and  both  of  the 
age  13  frames,  but  not  the  age  17  frame.     In  total,  the  five  frames  from  the 
94  PSUs  included  about  38,000  unique  schools,  and  each  school  with  a  grade  in 
the  range  1  through  12  was  included  on  at  least  one  frame. 


Table  3.2 

Grade  Definition  of  School  Eligibility  for  Frame  Inclusion 

Age  9  Age  13  Age  17 

Fall/Winter  (Bridge  A)      2-5  6-9 

Spring  1-4  5-8  9-12 


The  schools  sampled  for  a  grade/age  group  within  each  PSU  were  selected 
via  a  systematic  sample,  with  probabilities  proportional  to  assigned  measures 
of  size.     For  the  spring  samples,  roughly  equal  measures  of  size  were 
assigned  to  schools  containing  an  estimated  number  of  grade/age-eligible 
students  ranging  from  20  to  150  (for  grade  3/age  9),  or  20  to  200  (for  grade 
7/age  13  and  grade  11/age  17).     Schools  above  the  indicated  maximum  size  were 
selected  with  probabilities  proportional  to  the  estimated  number  of 
grade/age- eligible  students.     Schools  with  fewer  than  20  estimated  grade/age 
eligibles  were  assigned  considerably  lower  measures  of  size,  and  thus  reduced 
probabilities  of  selection,  since  they  had  considerably  higher  per- student 
administrative  costs.     Overall  probabilities  of  selection  for  high-minority 
schools  (those  where  more  than  10  percent  of  students  were  Black,  Hispanic, 
or  Asian)  were  twice  those  for  other  schools  to  enlarge  the  sample  for  such 
students,  thereby  enhancing  the  reliability  of  estimates  for  these  minority 
groups.     In  selecting  the  samples,  the  frames  were  sorted  within  PSU  by 
minority  status  and  the  measure  of  size. 

For  ages  9  and  13,  a  sample  of  schools  was  drawn  for  the  Bridge  A 
assessment  conducted  in  the  fall  and  winter.     These  were  then  excluded  from 
the  frame  when  the  sample  of  schools  was  drawn  for  the  spring  assessments. 
Within  each  PSU  for  each  age  class,  clusters  of  schools  were  formed,  with 
each  cluster  large  enough  to  provide  the  required  sample  size  of  students. 
One  cluster  was  then  selected  per  PSU  (except  in  the  Chicago  and  Los  Angeles 
MSAs ,  where  three  clusters  were  chosen). 

After  the  initial  sampling  of  schools  was  completed,   information  was 
obtained  to  update  the  sample  for  new  eligible  schools.     Public  school 
districts  and  dioceses  of  initially  selected   ichools  were  asked  to  give 
information  about  new  schools  and  schools  with  greatly  Increased  enrollment 
or  with  changes  in  grade  structure  within  their  district/diocese.  Similar 
information  was  sought  for  new  and  enlarged  private  schools  in  the  same 
districts/dioceses .     New  and  enlarged  schools  so  identified  were  given  an 
appropriate  chance  of  inclusion  in  each  of  the  samples  for  which  they  were 
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eligible.     Four  new  schools  were  sampled  this  way  in  total,  but  all  were 
found  ultimately  to  be  out-of -scope. 

In  PSUs  where  school  refusals  were  relatively  heavy  for  a  particular 
sample,  substitute  school  selections  were  made,  replacing  the  refusals  (to 
the  extent  feasible) ,  with  schools  from  within  the  same  PSU  and  similar  in 
size,  affiliation,  grade  span,  and  minority  composition.    This  procedure 
maintained  the  student  sample  sizes  needed,  keeping  variance  and  nonresponse 
bias  at  acceptable  levels.    Table  3.3  shows  the  number  of  schools  selected 
(and  in-scope),  cooperating,  and  substituted  in  each  of  the  school  samples 
The  cooperation  rates  given  are  based  on  the  initially  selected  sample  of 
schools,  and  schools  selected  only  for  the  Language  Minority  Probe  are 
excluded.     Note  that  since  the  response  rates  quoted  do  not  include  the 
substitute  selections,   the  potential  for  nonresponse  bias  is  likely  to  be 
somewhat  less  than  these  rates  would  indicate.     This  is  because  the 
substitute  selections  were  chosen  based  on  their  similarity  to  the  initially 
refusing  selections.  ^ 

Further  details  on  the  sampling  of  schools  are  given  in  section  2  4  of 
Burke  et  al.  (1987). 


Table  3.3 

School  Sample  Sizes,  Refusals,  and  Substitutes 


Grade 

3/Age  9 

Grade 

7/Age  13 

Grade  11/Age  17 

Winter 

Spring 

Fall 

Spring 

Spring 

Total 

Selected,  in-scope 

186 

494 

153 

560 

521 

1917 

No  eligible 

students  enrolled 

4 

5 

27 

62 

22 

120 
255 
91 

Refusals 

25 

52 

24 

61 

90 

Substitutes 

18 

20 

11 

18 

24 

Final  assessed 

sample 

175 

457 

113 

455 

433 

1633 

School  cooperation 

rate 

87% 

89% 

84% 

89% 

81% 

87% 

The  considerable  numbers  of  schools  sampled  with  no  eligible  students 
enrolled  resulted  primarily  from  the  fact  that,  for  example,  for  grade  7/age 
13    schools  with  grades  5,  6,  or  8  but  no  grade  7  were  sampled.     Such  schools 
had  a  reasonable  chance  of  containing  some  age  13  students.     Often  they  did 
have  a  number  of  eligible  students,  but  sometimes  they  had  none.  Such 
schools  account  for  the  sampled  schools  with  no  eligible  students  enrolled 
Because  of  the  grade  structure  of  schools,   this  occurred  most  often  for  age 
13.     These  response  rates  are  comparable  with  those  of  previous  assessments 
In  both  1982  and  1984  the  overall  cooperation  rate  of  schools  was  88  percent 
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3.3    ASSIGNMENT  OF  SESSIONS  TO  SCHOOLS,  BY  TYPE 


The  assignment  of  sessions  to  schools  served  as  the  third  stage  of 
sampling.     This  assignment  was  done  separately  by  the  three  tyr:  s  of 
sessions,  designated  spiral.  Bridge  A  (fall  and  winter),  and  Bridge  B 
(spring),  which  represent  separate  samples  of  the  student  population. 

The  Bridge  A  assessments  involved  three  distinct  booklets  each  for  ages 
9  (winter)  and  13  (fall).     Schools  to  participate  in  these  assessments  were 
selected  from  the  subsample  of  64  PSUs  which  had  been  designated  as  the 
Bridge  A  PSUs.     Each  of  the  three  distinct  booklets  for  an  age  group  was  to 
be  administered  within  each  of  the  PSUs.     To  avoid  the  possibility  that,  for 
a  given  PSU,  a  particular  bridge  session  might  be  assigned  only  to  a  school 
with  one  or  very  few  eligibles,  small  schools  were  clustered  with  other 
schools  in  the  same  PSU  to  form  clusters  of  a  specified  minimum  number  of 
eligibles.     At  each  age  bridge  sessions  were  then  assigned  to  schools  with 
probability  proportional  to  the  estimated  number  of  age  eligibles  within  the 
school  (or  school  cluster) . 

A  subsample  of  the  spring-selected  schools  were  allocated  Bridge  B 
sessions.     Bridge  B  consisted  of  two  tape-administered  sessions  (booklets  4 
and  5).     Bridge  B  was  conducted  in  all  94  sample  PSUs.     Most  schools 
allocated  to  a  Bridge  B  assessment  also  participated  in  the  spiral 
assessment . 

In  many  schools  selected  for  spring  assessment  a  number  of  students  were 
assessed  as  part  of  the  Language  Minority  Probe,  rather  than  for  spiral  or 
bridge  assessment.     Details  of  the  sample  design  for  the  Language  Minority 
Probe  are  given  in  Appendix  A  of  Burke  et  al.  (1987). 

Chapter  3  of  Burke  et  al.   (1987)  contains  fuller  details  on  the 
assignment  of  sessions  to  schools. 

3.4    THE  SAMPLES  OF  STUDENTS 

In  the  fourth  stage  of  sampling,  a  consolidated  list  of  all 
grade-eligible  and  age-eligible  students  was  established  for  each 
spring-selected  school.     Only  age  eligibles  were  listed  for  Bridge  A.  since 
there  was  no  assessment  of  students  eligible  by  grade  alone. 

For  spring-select  ad  schools  with  fewer  than  233  eligibles  for  grade 
11/age  17,  244  eligibles  for  grade  7/age  13,  and  166  eligibles  for  grade 
3/age  9) ,  all  eligible  students  were  invited  for  assessment  (some  of  them  for 
the  Language  Minority  Probe  in  some  schools).     Otherwise,  a  systematic  sample 
of  eligible  students  was  drawn  (about  210  for  grade  11/age  17,  220  for  grade 
7/age  13,  and  150  for  grade  3/age  9).     In  all  cases,  the  list  of  eligible 
students  was  prepared  by  the  school,  and  sampling  (if  any)  was  undertaken  by 
Westat  district  supervisors  using  prescribed  computer-prepared  instructions, 
tailored  to  each  individual  school,  and  based  on  the  estimated  number  of 
eligibles  in  the  school.     For  the  spring  assessment,  students  were  assigned 
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by  Westat  district  supervisors  to  spiral  or  Bridge  B  sessions,  using  a 
prescribed  probability  sampling  procedure. 

For  Bridge  A  assessments,  many  of  the  selected  schools  had  their  age 
eligibles  sampled  at  a  rate  of  one.     Age-eligible  students  in  large  schools 
in  selected  bridge  school  clusters  were  sampled  at  a  rate  calculated  to 
provide  the  target  sample  size. 


3.5    THE  SAMPLE  OF  EXCLUDED  STUDENTS 

Some  students  selected  for  the  sample  were  deemed  unassessable  by  the 
school  authorities  because  they  had  limited  English  language  proficiency, 
were  judged  as  being  educable  mentally  retarded,  or  were  functionally 
disabled.     In  these  cases,  an  excluded  student  questionnaire  was  filled  out 
by  the  school  staff,  listing  the  reason  for  excluding  the  student  and 
providing  some  background  information. 


3.6    STUDENT  PARTICIPATION  RATES 

Table  3.4  below  summarizes  the  rates  of  exclusion  and  participation  of 
invited  students  for  the  different  age  classes  in  1986.     Rates  were  very 
similar  to  those  experienced  in  1984.    Make-up  sessions  were  called  for  when, 
for  various  reasons,  more  than  a  tolerable  number  of  invited  students  did  not 
appear  for  the  originally  scheduled  assessment  sessions  in  a  school.  The 
participation  rate  gives  the  number  of  students  assessed  as  a  percentage  of 
the  number  invited.     Note  that  although  the  participation  rate  for  grade 
11/age  17  declined  somewhat  from  that  of  the  1984  assessment,  it  is  in  excess 
of  the  rate  the  rate  of  74.2  percent  attained  in  the  1981-82  assessment. 

Table  3.4 

Participation  and  Exclusion  Rates  by  Age  Class 
(Unweighted,  Excluding  Language  Minority  Probe) 

Participation  Participation 
Excluded  (X)     Number  Invited         Rate  (%)        Rate  a^--1984 

Grade  3/Age  9  3.9*  34,7^1  92.9 

Grade  7/Age  13  3.7*  42,641  89.2 

Grade  11/Age  17         3.4  55,309  78.9 


91.3 
87.3 
82.8 


*Exclusion  rates  given  are  for  spring  assessment  only.     Participation  rates 
include  both  spring  and  fall/winter  samples. 


3.7      THE  ASSOCIATED  TEACHER- STUDENT  SAMPLE 

The  teacher  questionnaire  was  administered  to  the  teachers  of  a  subgroup 
of  students  sampled  for  spiral  sessions.     The  purpose  of  this  sample  was  to 
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estimate  the  number  (proportion)  of  students  whose  teachers  had  various 
attributes,  not  to  estimate  the  attributes  of  the  teacher  population. 

It  was  assumed  that  at  grade  3/age  9  most  teachers  who  taught  reading 
also  taught  mathematics  and  science,  and  so  questions  covering  instruction  i 
all  three  subject  areas  were  included  in  the  questionnaire  for  this 
grade/age.    For  the  other  grade/ages  it  was  assumed  that  different  teachers 
would  be  teaching  each  of  the  subject  areas  covered  by  the  assessment.  Sine 
the  budget  did  not  permit  the  collection  of  information  from  teachers  from 
all  subject  areas  within  each  school,  a  sampling  scheme  was  devised.  One 
subject  area  was  designated  at  random  for  each  spiral-allocated  school.  As 
noted,  for  grade  3/age  9  the  subject  was  always  reading.     For  grade  7/age  12 
the  subject  area  selected  was  one  of  three:     reading,  mathematics,  and 
science.     For  grade  11/age  17  the  subject  area  was  one  of  four:  reading, 
mathematics,  science,  and  U.S.  history. 

A  sample  of  teachers  of  the  selected  subject  area  was  drawn  in  each 
school  in  the  following  manner.     For  each  spiral  session,  a  subsample  of 
students  was  selected  and  the  school  coordinator  was  asked  to  identify,  for 
each  such  student,  the  teacher  in  the  designated  subject  area  who  was 
teaching  the  student.     (For  reading,  English  or  language  arts  teachers  were 
selected.)    Up  to  five  teachers  per  school  were  selected  in  this  manner. 
These  instructors  completed  a  teacher  questionnaire. 

Chapter  4  of  Burke  et  al.   (1987)  describes  in  detail  the  steps  used  to 
obtain  the  various  student  and  teacher  samples. 
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CHAPTER  4 
Instrument  and  Item  Information 


Chapter  4 
INSTRUMENT  AND  ITEM  INFORMATION^ 


Janet  R.  Johnson 
Educational  Testing  Service 


The  1986  assessment  incorporated  five  distinct  instruments:  student 
assessment  booklets,  a  questionnaire  for  excluded  students,  a  teacher 
questionnaire,  a  computer  coordinator  questionnaire,  and  a  school 
characteristics  and  policies  questionnaire.     The  data  collected  from  these 
instruments  are  available  on  the  1986  NAEP  public-use  data  tapes.  This 
chapter,  which  describes  the  assessment  instruments,  begins  with  a  discussion 
of  how  items  were  organized  into  blocks  to  create  the  student  assessment 
booklets . 


4.1     STUDENT  ASSESSMENT  INSTRUMENTS 

Student  assessment  booklets  contained  both  cognitive  and  noncognitive 
Items.     Cognitive  items  were  used  to  assess  student  achievement  in  the 
subject  areas  of  reading,  mathematics,  science,  computer  competence    U  S 
history,  and  literature.     These  items  were  arranged  into  subject-specific 
blocks.     Noncognitive  items  asked  questions  related  to  students'  backgrounds 
and  attitudes.     Some  noncognitive  items  were  presented  to  every  student- 
these  were  placed  together  in  a  block  called  the  common  block  or  common 'core 
Other  noncognitive  items  were  attitude  items  specific  to  one  of  the  subject 
areas;  these  items  appeared  at  the  beginning  of  blocks  containing  cognitive 
items  related  to  the  same  subject  area. 

Each  student  at  all  three  grade/ages  was  administered  a  single  booklet 
containing  the  common  background  block  followed  by  three  subject-area  blocks 
The  composition  of  the  booklets  varied.     For  example,  the  balanced  incomplete 
block  (BIB)  spiralled  booklets  contained  either  0,  1,  2,  or  3  reading  blocks- 
the  remaining  blocks,   if  any,  were  in  mathematics,  science,  computer 
competence,  or,   in  the  cc^e  of  4  of  the  93  booklets  used  at  grade  11/age  17 
U.S.  history  and  literature.     Note:     The  reading  blocks  used  for  grade  7/age 
13  were  identical  in  every  respect  to  those  used  for  grade  11/age  17. 

Table  4.1  shows  the  total  number  of  blocks  created  for  each  subiect 

area . 


^Some  of  the  tables  for  this  chapter  were  generated  by  David  Freund  and 
Alfred  Rogers. 
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Table  4.1 

Number  of  Subject  Area  Blocks  Administered 


Subject  Area  Grade  3 /Age  9    Grade  7 /Age  13    Grade  11 /Age  17 

Reading  6  6  6 

Mathematics  7  9  11 

Science  7  9  11 

Computer  Competence  3  6  6 

U.S.  History  -  "  ^ 

Literature  -  -  ^ 

Total  23  30  42 


The  incomplete  BIB  design  of  these  blocks  generated  a  total  of  52 
booklets  for  grade  3/age  9,  68  booklets  for  grade  7/age  13,  and  93  booklets 
for  grade  11/age  17.     (Of  these,  one  booklet  at  each  age  was  specially 
constructed  for  use  in  the  Language  Minority  Probe.     The  booklet  contained  a 
special  block  of  background  questions  tailored  to  Language  Minority  students , 
followed  by  the  standard  common  background  block,  one  block  of  reading 
cognitive  items,  and  one  block  of  mathematics  cognitive  items.) 

Tables  A.l,  A. 2,  and  A. 3  in  Appendix  A  show  which  subject  area  blocks 
were  contained  in  each  booklet  for  each  grade/age.     Tables  A. 4,  A. 5,  and  A. 6 
in  Appendix  A  are  matrices  showing,  for  each  grade/age,   the  number  of  times 
each  block  is  paired  with  every  other  block  in  the  BIB  spiral  sample. 

The  1986  assessment  included  two  bridge  studies.     For  9-year-olds  and 
13-year-olds,  Bridge  A  measured  the  effect  of  changing  the  student  age 
definition  (from  calendar  year  to  school  year)  and  the  time  of  year  the 
assessment  was  conducted  (from  fall  and  winter  to  spring) .     Since  neither  age 
definition  nor  time  of  testing  was  changed  for  17-year-olds,  those  students 
were  not  included  in  the  Bridge  A  study.     Each  Bridge  A  student  took  a  single 
booklet  containing  a  mathematics  block,  a  reading  block,  and  a  science  block. 
(Booklet  configuration  is  shown  in  Table  4.2.)     The  same  booklet  was 
administered  to  an  entire  assessment  session.     The  mathematics  and  science 
blocks  were  paced  (presented  aurally  using  a  tape  recorder) .     The  tape 
recorder  was  turned  off  for  the  reading  block  in  each  session. 


Table  4.2 
Bridge  A  Booklet  Configuration 

Blocks 

Booklet  4  Age  9  Age  13 

1  9R1        9M1        9S1  13R1        13M1  13S1 

2  952        9R2        9M3  13S2        13R2  13M3 

3  9M2        9S3        9R3  13M2        13S3  13R3 


ERLC 


-78- 

87 


The  second  bridge  study,  Bridge  B,  measured  the  effect  of  changing  from 
tape-recorded  administration  to  print  administration.     Each  student  in  this 
bridge  sample  also  took  a  single  booklet  containing  three  blocks:     either  two 
science  blocks  and  one  mathematics  block,  or  two  mathematics  blocks  and  one 
science  block.    Booklet . configuration  is  shown  in  Table  4.3. 

Table  4.3 
Bridge  B  Booklet  Configuration 

Blocks 

Booklet  #  Age  9  Age  13  Age  17 

4  9M1      9M2      9S3         13M1      13M2      13S3         17M1      17M2  17S3 

5  9S1      9S3      9M3         13S1      13S2      13M3         17S1      17S2  17M3 


4.1.1  Timing 

The  length  of  time  allotted  for  each  block  changed  slightly  from  1984  to 
1986.     In  1984,  each  age  was  given  a  6 -minute  conmion  core  of  background  and 
attitude  questions  followed  by  three  subject  area  blocks  of  14  minutes  each. 
At  the  end  of  each  14-minute  interval,  the  students  were  told  to  move  to  the 
next  block.     Approximately  the  first  2  minutes  of  these  subject  area  blocks 
were  devoted  to  answering  background  questions  related  to  the  subject  area. 
In  1986,  13-  and  17 -year- olds  again  had  6  minutes  to  respond  to  the  common 
core  background  and  attitude  questions;  however,  for  9-year-olds,  the  common 
core  questions  at  the  beginning  of  each  booklet  were  read  aloud  to  them  and 
took  15  minutes  to  complete.     The  9 -year-olds  were  given  13  minutes  to  read 
and  respond  to  the  exercises  in  each  subsequent  block;  the  13-  and  17 -year- 
olds  were  given  16  minutes.     The  number  of  exercises  per  block  was  increased 
to  allow  for  the  amount  of  time  allotted  for  each  block.    With  the  exception 
of  one  mathematics  block  and  two  science  blocks  at  age  9,  all  subject  area 
blocks  at  all  ages  contained  some  background  and  attitude  questions. 

For  an  overview  of  the  composition  of  the  1986  assessment  blocks  see 
Tables  A. 7,  A. 8,  and  A. 9  in  Appendix  A.     Tables  A. 10,  A. 11,  and  A. 12  in 
Appendix  A  show,  by  grade/age,  the  booklets  in  which  each  item  block  appears. 

Many  items  in  each  subject  area  were  used  at  more  than  one  grade/age. 
Tables  A. 13  through  A. 18  in  Appendix  A  list  each  1986  cognitive  item  in  NAEP 
ID  order,  with  a  short  descriptor,  the  block  in  which  the  item  appears  for 
each  grade/age,  and  the  location  of  the  item  within  the  block.     Tables  A. 19 
through  A. 25  provide  the  same  information  for  each  noncognitive  item. 
Complete  item  text  is  available  on  the  microfiche  accompanying  the  public-use 
data  tapes . 
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4.2    EXCLUDED  STUDENT  QUESTIONNAIRE 


The  excluded  student  questionnaire  was  developed  and  used  for  the  first 
time  in  the  1984  assessment.     It  was  designed  to  gather  more  information 
about  particular  conditions  for  exclusion  and  characteristics  of  the  learning 
experience  of  excluded  students. 

The  questionnaire  was  completed  by  school  personnel  for  every  student 
who  was  selected  for  inclusion  in  the  NAEP  sample  but  was  unable  to  respond 
to  items  because  he  or  she  was  judged  by  school  personnel  to  be  non- English 
speaking,  educable  mentally  retarded,  or  functionally  disabled.     The  three- 
page  questionnaire  was  used  to  gather  information  concerning  special 
education,  language,  and  other  student  programs.     A  copy  of  the  excluded 
student  questionnaire  is  available  on  the  microfiche  accompanying  the  public- 
use  data  tapes. 

Of  the  119,137  students  sampled  for  the  1986  assessment,   5,209  were 
excluded  by  the  school  due  to  classification  as  educable  mentally  retarded, 
non-English  speaking,  or  functionally  disabled.     There  were  1,476  (4.4 
percent)  excluded  students  in  grade  3/age  9,  1,768  (4.4  percent)  in  grade 
7/age  13,  and  1,965  (4.3  percent)  in  grade  11/age  17. 


4.3     TEACHER  QUESTIONNAIRE 

The  teacher  questionnaire,  developed  and  used  for  the  first  time  in  the 
1984  assessment,  was  used  in  1986  to  gather  information  on  teachers  of 
various  subjects.     One  subject  from  the  list  in  Table  4.4  was  designated  for 
each  grade /age  assessment  in  each  school. 


Subjects 

Grade  3 /Age  9 
English/Language  Arts 


Table  4.4 
Taught  by  1986  Teacher 

Grade  7 /Age  13 

English/Language  Arts 

Mathematics 

Science 


Sample 

Grade  11 /Age  17 

English/Language  Arts 

Mathematics 

Science 

U.S.  History 


At  grade  3/age  9  the  English/language  arts  teacher  questionnaire  also 
contained  several  questions  about  mathematics  and  science  instruction  because 
it  was  assumed  that  these  younger  students  were  taught  all  subjects  in  a 
self-contained  classroom  situation. 

At  grade  7/age  13  and  grade  11/age  17,  the  multistep  sampling  process 
for  the  selection  of  teachers  to  answer  the  questionnaire  resulted  in 
students  being  introduced  to  as  many  as  five  teachers'  names  before  the 
assessment  began. 
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For  each  spiral  session,  a  subsample  of  students  was  selected  and  the 
school  coordinator  was  asked  to  identify,  for  each  selected  student,  the 
teacher  in  the  designated  subject  area  who  was  teaching  the  student!  These 
instructors  completed  the  teacher  questionnaire. 

Responses  were  received  from  a  total  of  774  3rd  grade  teachers    784  7th 
grade  teachers,  and  1,243  11th  grade  teachers.    A  copy  of  the  teacher 
questionnaires  is  available  on  the  microfiche  accompanying  the  public -use 
data  tapes . 

4.4  COMPUTER  COORDINATOR  QUESTIONNAIRE 

This  instrument  was  used  to  gather  information  about  subjects  aided  by 
computer  instruction,  computer  topics  and  courses  taught,  and  computer 
resources.     The  computer  coordinator  questionnaire  was  not  linked  to 
students.     The  questionnaire  was  completed  by  the  computer  coordinator,  if 
there  was  one,  for  each  school  included  in  the  spiral  assessment.     If  the 
school  did  not  have  a  full-  or  part-time  staff  member  who  coordinated  the  use 
of  computers  for  instruction  and/or  taught  about  computers,   then  no 
questionnaire  was  completed  by  that  school. 

Computer  coordinator  questionnaire  responses  were  received  from  232  of 
632  3rd  grade  schools,  373  of  568  7th  grade  schools,  and  328  of  433  Hth 
grade  schools . 

A  copy  of  the  computer  coordinator  questionnaire  is  available  on  the 
microfiche  accompanying  the  public-use  data  tapes. 

4.5  SCHOOL  CHARACTERISTICS  AND  POLICIES  QUESTIONNAIRE 

A  school  characteristics  and  policies  questionnaire  was  distributed  to 
each  participating  school  to  be  completed  by  either  the  school's  principal  or 
another  person  familiar  with  data  concerning  enrollment,  facilities, 
curricula,  and  staff  development. 

Responses  were  received  from  583  of  632  cooperating  3rd  grade  schools, 
521  of  568  cooperating  7th  grade  schools,  and  392  of  433  cooperating  11th 
grade  schools.     Cooperation  rates  were  88.7,   88.1,  and  82.7  respectively  for 
3rd,  7th,  and  11th  grade  schools;  the  overall  cooperation  rate  was  86.8 
percent. 

A  copy  of  the  school  characteristics  and  policies  questionnaire  is 
available  on  the  microfiche  accompanying  the  public-use  data  tapes. 
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Chapter  5 
FIELD  ADMINISTRATION 


Nancy  Caldwell  and  Renee  Slobasky 
Westat,  Inc. 


5.1  ORGANIZATION 

The  field  work  for  the  1986  National  Assessment  was  directed  by  the 
field  director  and  assistant  field  director(s)  in  Westat's  home  office. 
Reporting  to  them  were  district  supervisors,  each  responsible  for  contacting 
districts  and  schools  and  conducting  the  assessments  in  their  geographic 
region.     Each  district  supervisor  hired,  trained,  and  supervised  the  work  of 
local  exercise  administrators,  who  conducted  the  assessment  sessions.  ETS 
home  office  and  regional  staff  supported  the  district  supervisors,  responding 
to  requests  for  technical  information  about  the  assessment  and  working  to 
convert  schools  and  districts  that  refused  to  participate  in  the  assessment. 

Because  the  spring  assessment  in  1986  differed  in  so  many  ways  from  the 
fall  and  wintej,  the  field  organization,  staffing  and  materials  were  also 
very  different  .     During  the  fall  and  winter,  16  district  supervisors  were 
responsible  for  all  assessment  activities  in  64  PSUs.     During  the  spring  the 
number  of  PSUs  increased  to  94.     The  workload  within  each  spring  PSU  also 
increased  substantially  because  all  three  grade/age  groups  were  assessed  at 
the  same  time.     Recognizing  this  increased  workload,  Westat  expanded  the 
district  supervisory  staff  substantially  from  16  (plus  two  backup 
supervisors)  to  54. 

The  district  supervisors  were  responsible  for  the  following  tasks  in 
each  of  the  areas  assigned  to  them: 

contacts  with  districts  and  schools; 

hiring  and  training  local  exercise  administrators; 

selecting  the  sample  of  students  to  be  assessed  in  each 
school  following  the  detailed  sampling  specifications 
provided  by  Westat; 

making  all  arrangements  with  the  schools; 
supervising  the  conduct  of  the  assessment; 


"^See  section  1.2.1  for  a  discussion  of  the  fall,  winter,  and  spring 
assessment  schedules  and  the  grade/age  groups  assessed  during  these  periods. 
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distributing  and  collecting  assessment  questionnaires; 

coir.pleting  all  administrative  records  and  forms;  and 

preparing  and  shipping  completed  assessment  materials  to 
ETS  and  Westat. 

The  district  supervisors  were  trained  by  home  office  staff  before  they 
began  their  work  in  the  field.  Each  supervisor  then  trained  his  or  her  own 
exercise  administrators  to  work  in  the  local  areas. 

5.2    OBTAINING  DISTRICT-  AKD  SCHOOL-LEVEL  COOPERATION 

During  the  stimmer  of  1985,  ETS  made  preliminary  contacts  with  the  states 
and  districts  that  had  schools  in  the  NAEP  sample.     These  initial  letters 
were  followed  by  telephone  and  in-person  contacts  by  the  district  supervisors 
and  ETS  staff  in  the  early  fall.     Introductory  meetings  were  held  with  school 
and  district  staff  to  describe  assessment  procedures  and  make  preliminary 
arrangements  for  the  assessment  in  each  school. 

The  results  of  the  school  contacting  phase  of  the  project  are  discussed 
in  detail  in  the  final  report  on  sampling  and  weighting  procedures  (Burke, 
Braden,  Hansen,  Lago ,  6c  Tepping,  1987,  pp.  2-15).     To  summarize,  there  was  a 
total  of  2,309  schools  originally  selected  to  participate  in  the  fall, 
winter,  and  spring  assessments,  including  the  Language  Minority  Probe.  Of 
these,  78  were  closed  or  out  of  scope,  leaving  2,231  eligible  schools. 
Cooperating  schools  (including  133  schools  with  no  eligible  students 
enrolled)  numbered  1,927  (86.4  percent).     There  were  304  refusals. 
Cooperation  was  slightly  higher  among  schools  selected  for  the  regular  NAEP 
sample  than  for  those  selected  for  the  Language  Minority  Probe  (86.8  percent 
versus  83.6  percent).     Further  details  on  cooperation  among  schools  excluding 
those  selected  solely  for  the  Language  Minority  Probe  are  given  in  Table  3.3 
of  Chapter  3. 

Assessments  were  conducted  in  a  total  of  1,887  schools.     Of  these,  1,794 
were  cooperatinp  schools  in  the  original  sample  that  had  eligible  students 
enrolled.     The  remaining  93  schools  were  replacements  for  refusals. 


5.3    SELECTING  THE  SAKPLE  OF  STUDENTS  TO  BE  ASSESSED 

The  schools  were  given  instructions  for  preparing  lists  of  age  and  grade 
eligible  students.     Because  of  the  Language  Minority  Probe,   schools  in  the 
spring  assessment  were  asked  to  prepare  three  lists:     one  of  eligible 
Hispanic  students,  one  of  eligible  Asian  American  and  American  Indian 
students,  and  one  of  all  other  eligible  students.     Prior  to  the  assessment 
dote,  the  district  supervisor  visited  the  schools  to  select  the  sample  and  to 
make  final  arrangements  for  the  assessment.     Supervisors  received  school- 
specific  sampling  instructions  from  Westat' s  home  office,  which  they  followed 
to  select  the  sample  of  students  to  be  invited  to  the  assessment.     If  the 
number  of  eligible  students  in  a  particular  school  was  very  different  from 
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what  was  anticipated,  the  district  supervisor  called  the  home  office  for 
revised  instructions. 

Once  the  sample  of  students  was  selected,  final  arrangements  could  be 
made  with  the  school  for  the  assessment  of  those  students.  Arrangements 
varied  depending  on  factors  that  included  the  number  of  students  to  be 
assessed,  the  availability  of  space  in  the  school,  and  the  school's  schedule 
as  well  as  other  considerations.     Since  the  BIB-spiral  booklets  are  self- 
administered.  there  was  considerable  flexibility  in  setting  up  the  assessment 
sessions • 


5.4    CONDUCT  OF  THE  ASSESSMENT 

Each  assessment  session  took  a  little  over  one  hour  to  conduct,  although 
there  was  some  variation  depending  on  how  quickly  the  students  arrived  for 
the  session  and  how  many  booklets  had  to  be  distributed.     An  introduction  and 
directions  were  read  by  the  NAEP  staff  following  a  prescribed  script  The 
supervisor  and  exercise  administrators  monitored  the  sessions  to  make  sure 
that  students  were  working  on  the  appropriate  sections  of  the  booklets 
Quality  control  visits  were  conducted  by  ETS  and  Westat  home  office  staff  to 
ensure  that  the  assessment  sessions  were  being  administered  correctly  and 
uniformly  by  field  staff.  ^ 

■  previous  cycles  of  the  assessment,  the  number  of  students  assessed 

varied  by  age  group.     From  among  the  students  invited  within  participating 
schools,  the  percent  assessed  was  highest  for  grade  3/age  9  (92  5  percent? 
next  highest  fgr  grade  7/age  13  (89.2  percent)  and  lowest  for  grade  ll/age'l7 
ToZ  ?r?ab^e  S.l"^*"'  ''''  '"'"^  experiences  are  compared  in  Chapter  I  and 

Table  5.1 

Comparison  of  1986  and  1984  Student  Participation  Rates 

by  Grade /Age 

Participation  Rate 
Grade /Age  igSg  1934 

Grade  3/Age  9  92.9  91.3 

Grade  7/Age  13  89.2  87  3 

Grade  11/Age  17  /8.9  82.8 


5.5     REPORT  ON  THE  FIELD  ADMINISTRATION 

A  thorough  discussion  of  the  field  organization  and  operations  is 
presented  m  the  Westat  Report  on  Field  Operations  and  n^r^  Collection 
Activities.  NAEP-Year  17  ri985-8M   (r..^A^.^^  x.  oi^u^.w  ^ngg)^ 

2 

Includes  Bridge  A  for  13-year-olds  and  9-year-olds. 
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CHAPTER  6 
Materials  Processing  and  Database  Creation 


Chapter  6 

MATERIALS  PROCESSING  AND  DATABASE  CREATION 


John  L.  Barone 
Educational  Testing  Service 


The  following  chapters  detail  the  receipt,  processing,  and  final 
disposition  of  the  assessment  materials  at  ETS  as  they  were  transcribed  to 
computer- readable  form  and  placed  in  an  integrated  NAEP  database  to  be  used 
for  data  analysis  and  reporting.     This  database  is  now  available  to  external 
users  via  the  public-use  data  tapes  (see  Rogers,  Kline,  Norris,  Johnson, 
Mislevy,  Zwick,  Barone,  &  Kaplan,  1988). 

The  scope  of  the  effort  required  to  perform  this  aspect  of  the  1986 
assessment  is  evidenced  by  the  following  numbers.     For  the  1986  assessment, 

more  than  150.000  assessment  booklets  were  received  and 
processed. 

This  processing  included: 

optically  scanning  more  than  3  million  double-sided  pages; 

professionally  scoring  more  than  700.000  student  responses  on 
240  open-ended  items; 

manually  key-entering  and  verifying  more  than  16.000 
assessment  booklets; 

using  the  NAEP  minicomputer-based  transcription  system  to 
track,  audit,  edit,  and  resolve  more  than  28  million 
characters  of  information; 

for  quality  control,  selecting  and  comparing  more  than  160.000 
characters  of  transcribed  data  to  the  actual  responses  in 
assessment  booklets; 

cataloging  more  than  2  million  characters  of  information 
on  a  total  of  4,800  assessment  items  and  derived 
variables,  as  part  of  a  comprehensive  item  information 
database ; 

developing  a  public -use  data  tape  package  containing  more 
than  150  million  characters  of  useful  information. 

These  numbers  alone  indicate  the  staggering  size  of  1986  NAEP  materials 
processing  and  database  operations.    However,  the  full  extent  of  this  effort 
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becomes  clearer  when  one  considers  that  over  90  percent  of  the  data 
transcription  activities  described  in  this  chapter  were  started  and  completed 
within  a  six-month  period,  with  a  conservatively  estimated  accuracy  rate  of 
fewer  than  2.5  errors  for  every  10,000  characters  of  information  transcribed. 

Materials  processing  and  database  creation  for  the  1986  assessment 
closely  paralleled  the  processes  used  in  the  1984  assessment.     This  allowed 
the  use  of  in-place,  proven  operational  procedures  and  computer  systems, 
which  will  be  highlighted  throughout  the  following  chapters. 

A  major  improvement  in  the  1986  assessment  was  the  introduction  of 
scannable  booklets  and  the  concomitant  shift  of  most  data  transcription  from 
manual  key  entry  to  computerized  scanning.     The  NAEP  systems  were  adapted  to 
accommodate  scanning  technologies  and  procedures  and  were  expanded  to  perform 
remote  computer  processing  via  network  technologies. 

In  past  NAEP  assessments,  each  of  the  three  grade/age  groups  were 
assessed  at  different  times  during  the  school  year.     This  allowed  the  data 
transcription  to  occur  over  a  nine-month  period.     In  the  main  1986 
assessment,  NAEP  assessed  all  three  grade/age  groups  in  the  spring.     The  high 
volume  of  input  combined  with  a  much  shorter  time  period  for  processing 
precluded  manual  entry  and  verification  of  student  booklets.     For  this 
reason,  the  1986  assessment  booklets  were  designed  to  be  read  by  computer 
scanning  devices,  and  the  NAEP  data  transcription  systems  were  modified  to 
accept  the  output  of  the  scanning  devices. 

The  flow  of  materials,  creation  of  data  files,  and  creation  of  the  NAEP 
database  are  depicted  as  an  ordered  set  of  processes  that  are  applied  either 
to  the  assessment  materials  or  to  the  transcribed  data.     The  following 
chapters  describe  each  of  these  processes  in  detail. 

The  large  volume  of  collected  data  and  the  complexity  of  the  NAEP 
design,  with  its  spiralled  distribution  of  many  booklets,   required  the 
development  and  use  of  NAEP-specif ic  data  entry  and  management  systems, 
including  carefully  planned  and  well-defined  editing,  quality  control,  and 
auditing  procedures.     This  chapter  discusses  the  original  1984  design  and 
implementation  of  these  systems,  and  the  adaptation  and  use  of  these  systems 
and  processes  as  applied  to  the  1986  assessment.     The  results  were  effective, 
resporsive  data  management  procedures  that  ensured  the  quality  and  integrity 
of  NAEP  data,  and  a  NAEP  database  that  met  the  original  objectives  of 
integrity  and  usefulness,  exceeding  stringent  standards  for  "correctness"  and 
quality. 

Figure  6.1  is  a  flow  diagram  that  shows  the  conceptual  framework  of 
ordered  processes  that  were  applied  to  the  NAEP  materials  and  data  files.  The 
dashed  line  through  the  center  of  the  figure  divides  the  outline  into  two 
seLS  of  processes,  processing  assessment  materials  and  database  creation, 
described  below. 

The  processes  represented  by  solid- line  boxes  in  the  flow  diagram  were 
performed  at  ETS  on  the  paper  materials  or  computer  files.     The  three 
processes  enclosed  in  dashed-line  boxes  (sample  of  schools,  field 
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Figure  6.1 
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administration,  and  derive  sampling  weights)  were  performed  by  Westat  and  are 
discussed  respectively  in  Chat^ters  3,  5,  and  14.     Two  Westat  reports,  the 
Report  on  Field  Operations  and  Data  Collection  Activities,  NAEP— Year  17 
(1985-86)   (Caldwell  6c  Slobasky,  1988)  and  National  Assessment  of  Educational 
Progress --17th  Year  Sampling  and  Weighting  Procedures.     Final  Report  (Burke, 
Braden,  Hansen,  Lago,  &  Tepping,  1987),  discuss  these  processes  in  detail. 

6.0.1    PROCESSING  ASSESSMENT  MATERIALS 

The  left  side  of  Figure  6.1  depicts  the  flow  of  NAEP  "paper"  materials. 
Chapter  6.1  describes  this  flow  in  detail  and  discusses  how  information 
contained  on  the  field  rosters,  schedules,  and  worksheets  were  used  as 
controlling  mechanisms  for  processing  of  materials.     The  figtire  follows  the 
path  of  each  assessment  instrument  (student  assessment  booklets,  school 
characteristics  and  policy  questionnaires,  computer  coordinator 
questionnaires,  teacher  questionnaires,  and  excluded  student  questionnaires), 
absentee  rosters,  school  worksheets,  and  administration  schedules  as  they  are 
tracked  through  the  appropriate  processes  that  result  in  the  final  integrated 
NAEP  database. 

The  following  is  a  brief  description  of  the  materials  processing 
activities  as  shown  on  the  left  side  of  Figure  6.1.     Each  description  refers 
the  reader  to  the  section(s)  or  chapter(s)  in  which  the  process  is  discussed 
in  detail. 

Field  Administration  is  the  conduct  and  monitoring  of  the  NAEP 
assessment  in  the  schools.     Chapter  5  summarizes  this  process. 

Materials  Receipt  refers  to  receipt  and  processing  of  assessment 
materials  at  ETS .     Section  6.1.1  describes  the  procedures  and  forms  that  were 
used  to  check  and  verify  the  receipt  of  documents  from  the  field.     It  also 
discusses  the  follow-up  procedures  that  were  initiated  when  discrepancies 
were  identified  and  the  subsequent  batching  of  NAEP  materials  for  further 
processing  and  data  transcription. 

Professional  Scoring  is  the  process  that  result;ed  in  the  scoring  of  the 
open-ended  NAEP  reading,  mathematics,  science,  and  computer  competence  items. 
Chapter  6.2  describes  the  items,  types  of  scoring  used,  scoring  operation, 
reliability  checks,  and  resolution  of  scoring  discrepancies.     Entry  and 
editing  of  this  data  are  discussed  in  sections  6.1.4  and  6.4.2. 

Data  Tra  <;cription  Systems  refers  to  the  methodology  used  to  transcribe 
NAEP  materials     nto  computer- readable  form.     The  transcription  method  used 
for  each  type  of  NAEP  instrument  is  discussed  in  Chapter  6.1.     Chapter  6.3 
describes  the  design,  structure,  and  development  of  the  NAEP -specif ic  data 
entry  system  used  to  transcribe  most  of  the  NAEP  materials  to  computer  files; 
it  also  discusses  the  tracking  and  audit  mechanisms  that  were  built  into  the 
system  to  ensure  that  all  data  was  properly  processed  and  accounted  for. 
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Originally  implemented  for  the  1984  assessment,  NAEP's  data 
transcription  system  has  proven  to  be  accurate,  efficient,  and  flexible.  It 
began  in  1984  by  using  manual  key  entry  and  verification  as  the  prime  vehicle 
for  data  entry.     In  the  1986  assessment  it  was  modified  to  accept  scannable 
booklets  as  the  main  source  of  input,  while  retaining  the  manual  entry  and 
verification  component  for  materials  that  were  not  designed  as  scannable 
documents  as  well  as  for  scannable  documents  that  for  some  reason  (e.g., 
torn)  could  not  be  read  by  the  scai»ner. 

Editing  refers  to  the  ETS  procedures  that  ensured  the  correctness  and 
integrity  of  the  NAEP  data  files  by  (1)  validating  every  field  of  NAEP  data 
that  was  entered  into  computer- readable  form,   (2)  identifying  any  invalid  or 
inconsistent  values,  and  (3)  correcting  or  flagging  as  unresolvable  those 
values  identified  as  invalid  or  inconsistent.     Chapter  6.4  describes  these 
procedures . 

Quality  Control  refers  to  the  ETS  procedures  that  assessed  the  accuracy 
of  the  data  transcription  and  editing  operations.     Chapter  6.5  discusses  the 
quality  control  procedures  used  in  NAEP  data  processing  and  provides  a 
summary  of  the  likely  error  rates. 

Materials  Storage  refers  to  the  final  disposition  of  NAEP  paper 
materials  after  processing  had  been  completed.     Chapter  6.1  discusses 
materials  storage. 


6.0.2    Database  Creation 

The  right  side  of  Figure  6.1  depicts  the  evolution  of  the  integrated 
NAEP  database  from  the  transcribed  data  to  the  final  database,  available  to 
external  users  via  the  public-use  data  tapes.     Chapter  6.6  describes  the 
processes  through  which  the  database  evolved. 

The  remainder  of  this  section  contains  a  brief  description  of  each 
process  involved  in  database  creation  as  shown  on  the  figure.  Each 
description  also  refers  the  reader  to  the  section(s)  or  chapter (s)  in  which 
the  process  is  discussed  in  detail. 

Sample  of  Schools  refers  to  the  process  performed  by  Westat  to  select 
the  schools  to  be  included  in  the  assessment.     This  process  is  discussed  in 
Chapter  3. 

Data  Files  refers  to  (1)  the  data  files  created  by  the  ETS/NAEP  data 
transcription,  editing  and  resolution  systems  and  (2)  the  labeling  files 
(discussed  in  Chapter  6.6)  that  contain  descriptive  information  on  every  item 
used  in  NAEP. 

Extract  is  the  process  discussed  in  section  6.6.1  that  created  data 
files  containing  specific  demographic  data  fields  from  the  ETS/NAEP  data 
files.     These  data  files  were  required  by  Westat  to  derive  sampling  weights. 
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Sample  Weights  Derivation  was  performed  by  Westat  and  is  discussed  in 
Chapter  14.  This  process  produced  computer  tape  files  containing  sampling 
weights  for  every  student  and  school  assessed  by  NAEP. 


Merge  refers  to  the  final  integration  of  NAEP  data  files  into  the  NAEP 
database.     This  process,  discussed  in  section  6.6.2,  merged  the  NAEP  data 
files,  labeling  files,  and  the  NAEP  sampling  weights  into  one  inclusive 
database . 

NAEP  Database  is  the  final,   integrated  NAEP  database  that  contains  all 
1986  NAEP  data.     This  is  the  database  that  is  ultimately  made  available  to 
external  users  via  the  public-use  data  tapes.     The  structure  of  the  internal 
NAEP  database  is  discussed  in  Chapter  6.6;  the  public-use  data  tapes,  which 
contain  all  of  the  nonconfidential  data  fields  from  the  internal  database, 
are  discussed  in  Chapter  6.7. 
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Chapter  6.1 
PROCESSING  ASSESSMENT  MATERIALS 


Alfred  M.  Rogers  and  Norma  A.  Norrls 
Educational  Testing  Service 


This  chapter  describes  the  procedures  through  which  NAEP  instruments 
schedules,  and  worksheets  were  received  at  ETS ,  and  the  methods  used  in  the 
subsequent  scoring,  scanning,   loading,  editing,  and  resolution  of  NAEP  data. 

6.1.1    RECEIPT  OF  MATERIALS 

It  was  the  responsibility  of  the  district  supervisor  to  complete  and 
mail  a  postcard  to  ETS  at  the  completion  of  assessment  administration  in  each 
school.     This  card  contained  the  assessed  school  identification,  the  number 
ot  boxes  shipped,  and  the  mode  of  shipment.     The  receipt  of  this  card  at  ETS 
alerted  staff  to  expect  arrival  of  the  shipment  within  seven  working  days 
If  after  seven  days  the  shipment  had  not  arrived,  ETS  notified  Westat    who  in 
turn  initiated  a  trace  of  the  shipment.     This  tracing  process  was  successful 
in  all  cases  except  one,   in  which  the  full  set  of  assessment  materials  from 
one  school  was  never  recovered.     Some  other  shipments  broke  open  in  transit. 
In  all,   56  booklets  were  lost  or  damaged. 

The  shipment  from  each  school  contained  the  school  worksheet- 
administration  schedule;  questionnaire  roster;  school,   teacher,  aAd  excluded 
student  questionnaires;  and  assessment  booklets,  bundled  by  session.  The 
format  and  content  of  these  instruments  are  documented  in  the  Westat  Report 
on  Field  Operations  and  Data  Collection  Activities.  NAEP- -Year  17  ngSS-Sftt 
(Caldwell  &  Slobasky,   1988).     The  following  discussion  of  check-in  procedures 
presumes  an  understanding  of  information  contained  in  and  interrelationships 
among  these  instruments. 

The  school  worksheet  contained  summa::y  counts  of  the  booklets  used  in 
all  assessment  sessions  in  each  school.     The  booklets  used  within  each 
session  were  counted  and  checked  against  the  count  written  on  the  school 
worksheet.    All  discrepancies  in  the  counts  were  referred  to  the 
administration  schedules  for  resolution.     The  booklet  numbers  from  the  bundle 
in  question  were  compared  against  the  listing  of  booklet  numbers  on  the 
schedule      If  the  discrepancy  could  not  be  resolved  by  this  process,  Westat 
was  notified,  who  in  turn  contacted  the  appropriate  district  supervisor  for 
resolution. 

Two  codes,   the  session  code  .and  the  batch  code,  were  then  assigned  to 
each  column  on  the  worksheet  and  to  the  corresponding  bundle  of  booklets  The 
two-digit  session  code  distinguished  spiral  from  bridge  sessions  and  regular 
xrom  makeup  sessions.     Codes  1  through  10  identified  regular  spiral  sessions- 
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code  11  was  used  for  makeup  spiral  sessions;  codes  21  through  25  uniquely 
identified  regular  bridge  sessions,  with  the  second  digit  corresponding  to 
the  booklet  number;  and  codes  31  through  35  similarly  identified  the  makeup 
bridge  sessions. 

The  use  of  a  batch  identification  code  was  necessitated  by  the 
introduction  of  machine -scannable  documents  in  this  assessment.  A 
preprinted,  scannable  header  sheet  was  attached  to  each  bundle  of  student 
booklets  to  be  used  to  identify  it  through  all  subsequent  scoring,  scanning, 
entry,  and  resolution  processing.     This  batch  header  sheet  was  pregridded 
with  a  unique  four-digit  sequence  code.     As  each  header  sheet  was  drawn  from 
the  pile,   it  was  gridded  with  the  age  group  code,   the  school  and  session 
codes,   the  current  batching  date,  and  the  number  of  booklets  to  be  processed. 
The  age  group  code  was  either  "N" ,   "T" ,  or  "S"  corresponding  to  the  9-,  13-, 
or  17 -year-old  cohorts.     The  batch  identification  code,  which  consisted  of 
this  age  group  code  and  the  sequence  number  from  the  header  sheet,  was  then 
written  at  the  bottom  of  the  session  information  on  the  school  worksheet. 


The  teacher  and  excluded  student  questionnaires  were  then  counted  and 
compared  against  the  questionnaire  roster.     All  discrepancies  in  the  teacher 
and  excluded  student  questionnaire  counts  were  referred  to  Westat  and  again, 
in  turn,  to  the  district  supervisor  for  resolution.     Field  administration 
procedures  permitted  a  separate  shipment  of  teacher,  excluded  student,  and 
school  questionnaires.     The  questionnaire  roster  listed  questionnaires  not 
included  in  the  shipment,  alerting  the  receiving  staff  to  expect  a  later 

shipment.  ^ 

If  the  supervisor  was  unable  to  collect  the  questionnaires  on  the  day  of 
the  assessment,  a  pre -addressed  envelope  was  left  at  the  school  so  that  the 
school  coordinator  could  mail  the  questionnaires  directly  to  ETS .     There  was 
no  other  follow-up  activity  to  obtain  uncollected  questionnaires  from  school 
personnel;  efforts  to  encourage  school  cooperation  were  focused  primarily  on 
student  assessment  activities. 


When  all  of  the  student- related  materials  for  a  school  had  been  received 
and  checked  in,  the  assessment  schedules,  school  worksheet,  assessment 
booklets,  and  questionnaires  were  forwarded  to  the  data  operations 
coordinator  for  transcription  processing.     The  operations  coordinator 
separated  these  materials  according  to  the  appropriate  data  entry  procedures: 
the  administration  schedules  were  accuimulated  and  shipped  in  batches  to  key 
entry;  the  school  worksheet  and  excluded  student,   teacher,  and  school 
questionnaires  were  sent  directly  to  data  entry  systems.     School  worksheets 
were  entered  ii\to  the  data  entry  system  on  a  daily  basis.  Questionnaires 
were  batched  and  held  for  data  entry  until  scheduling  permitted.  Assessment 
session  bundles  were  forwarded  to  the  professional  scoring  area  where  open- 
ended  items  were  scored.     When  scoring  of  the  open-ended  items  was  completed, 
the  assessment  session  bundles  were  shipped  to  the  optical  scanning 
department. 
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6.1.2    ADMINISTRATION  SCHEDULES 


The  administration  schedules  contain  the  demographic  characteristics  of 
the  students  selected  for  the  assessment.    This  information,  which  included 
the  sex,  ethnic  origin,  grade,  and  birthdate  of  the  sampled  students  was 
used  by  Westat  in  the  derivation  of  sampling  weights.    The  booklet  numbers  of 
the  students  who  participated  were  transferred  to  the  schedule  at  the  time  of 
the  assessment,  and  the  demographic  information  was  in  turn  transferred  to 
the  front  covers  of  the  booklets  after  the  assessment. 

The  demographics  of  the  students  who  were  sampled  but  did  not 
participate  in  the  assessment  (exclusions  -and  absentees)  were  used  to  adjust 
the  sampling  weights  of  those  who  did.     The  excluded  student  information 
could  be  obtained  from  the  excluded  student  questionnaire  data,  but  the 
information  on  absentees  could  only  be  found  on  the  administration  schedules 
It  was  therefore  necessary  to  transcribe  this  information  to  computer- 
readable  media  and  combine  it  with  the  assessed  and  excluded  student  data. 

The  administration  schedule  data  was  transcribed  to  computer  tape  by  the 
key  entry  systems  at  ETS.     One  record  was  generated  for  each  absent  student 
(line)  on  the  form.     The  PSU.  school,  and  session  codes  from  the  top  of  the 
form  were  repeated  for  each  student  on  the  form.     The  information  transcribed 
for  each  absent  student  included  sex,  grade,  and  birthdate.     These  data  were 
ultimately  used  by  Westat  to  adjust  the  sample  weiehts. 

At  the  completion  of  entry  processing,  the  keyed  data  tape  was  copied  to 
a  disk  file  for  editing  and  quality  control  processing.     The  editing  process 
consisted  of  a  validation  program  and  an  interactive  text  editor  for 
:orrecting  erroneous  data.     The  validation  program  checked  that  the 
demographic  information  was  present  and  within  the  appropriate  ranges  The 
schedules  were  referred  to  during  this  process  for  the  resolution  of  any 
errors  or  discrepancies  uncovered  by  the  program  and  to  "spot-check"  records 
for  quality  control. 

The  assessment  schedules  were  retained  by  the  operations  coordinator  in 
anticipation  of  future  questions  about  and  references  to  the  sample  This 
proved  to  be  the  most  efficient  and  compact  means  of  retaining  the  relevant 
raw  data  since  the  schedules  for  all  three  grade/age  assessments  could  be 
contained  in  three  storage  boxes. 


6.1.3    SCHOOL  WORKSHEETS 

Each  column  of  the  school  worksheet  contained  information  pertaining  to 
the  administration  activity  of  each  session  within  a  school  This 
information  included  the  date,  time,  and  location  of  the  administration  the 
exercise  administrator  code,  and  the  counts  of  the  students  sampled,  absent 
and  assessed.     Additionally,  each  column  contained  a  session  code  and  batch' 
identification  code  that  were  written  in  by  receipt  processing  st-ff  This 
information  was  entered  into  the  system  by  selecting  the  first  option  on  the 
data  entry  menu  (Figure  6.1.1). 


-101- 


105 


The  worksheet  entry  program  received  its  input  through  two  entry 
screens.     The  first  entry  screen  (Figure  6.1.2)  requested  school-level 
information:     the  PSU  and  school  codes  and  the  total  number  of  sessions  to  be 
entered  for  that  school.     This  count  was  further  broken  down  into  the  four 
types  of  sessions:     regular  spiral,  makeup  spiral,  regular  bridge,  and  makeup 
bridge.     The  program  would  then  display  the  second  entry  screen  (Figure 
6.1.3)  once  for  each  session,  requesting  the  session-level  information.  When 
all  sessions  for  a  school  had  been  entered,  the  program  would  redisplay  the 
first  entry  screen,  ready  to  process  the  next  worksheet.     The  operator  could 
either  enter  new  information  or  press  ENTER  to  return  to  the  main  menu. 


Figure  6.1.1 


Data  Entry  System  Main  Menu 


NAEP/COD 


SYSTEM 


MENU 


OPTION: 


1 


School  Worksheet  Entry 


2 


Load  Scanning  Tape 


3 


Student  Data  Entry /Verification/Resolution 


4 


Questionnaire  Data  Entry /Verification/Resolution 


X 


Quit 


Enter  Option  Code: 
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Figure  6.1.2 
School  Worksheet  Entry  Screen  #1 


SCHOOL  WORKSHEET 


PSU  #: 
SCHOOL  #: 


TOTAL  NUMBER  OF  SESSIONS: 

NUMBER  OF  SPIRAL  SESSIONS  (0-10): 
NUMBER  OF  MAKEUP  SPIRAL  SESSIONS  (0-10): 
NUMBER  OF  ORIGINAL  TAPE  SESSIONS  (0,1,2): 
NUMBER  OF  MAKEUP  TAPE  SESSIONS  (0,1,2): 


Figure  6.1.3 
School  Worksheet  Entry  Screen  #2 


SCHOOL  WORKSHEET 

PSU  #:   

SCHOOL  #:   

TAPE/SESSION  #:   

DATE         ■  ' 
TIME 

EA'S  INITIALS 
EA'S  ID 


#  TO  BE  ASSESSED 
#  ASSESSED 
#  ABSENT 

BATCH  NUMBER: 
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The  entry  system  controlled  the  proc3ssing  of  student  data  and 
maintained  statistics  on  the  entry  activity  at  the  session  level.     This  was 
accomplished  by  means  of  a  tracking  file,  each  record  of  which  contained  all 
control  and  reporting  information  for  one  session.     The  entry  of  the  school 
worksheet  information  thus  generated  a  new  record  on  the  tracking  file  for 
each  session,  initializing  the  control  parameters. 

The  operations  coordinator  was  provided  with  procedures  for  periodically 
monitoring  and  reporting  data  entry  activity.     These  procedures  compared  the 
counts  of  booklets  processed  at  each  stage  with  the  initial  counts  from  the 
worksheet,  and  flagged  discrepancies.     This,   in  turn,  alerted  the  coordinator 
to  possible  missing  or  extra  booklets.     If  the  school  worksheet  information 
was  determined  to  be  in  error,   the  operations  coordinator  had  the  facility  to 
correct  the  tracking  file  data  to  prevent  reappearance  of  the  discrepancies 
in  the  activity  report. 

The  school  worksheets  were  retained  by  the  operations  coordinator  in 
anticipation  of  later  queries,  since  they  could  be  conveniently  stored  and 
easily  referenced. 

6.1,4     STUDENT  ASSESSMENT  INSTRUMENTS 

The  student  assessment  booklets  were  forwarded  directly  to  the  scoring 
area  as  the  complete  set  of  materials  was  received  from  each  school.  The 
booklets  were  batched  by  session,  with  a  batch  header  sheet  attached  to  the 
top  of  each  bundle.     This  preprinted,  &<;annable  sheet  contained  the  PSU, 
school,  and  session  codes,  and  a  unique  batch  identification  code  serving  to 
identify  each  batch.     The  header  sheets  were  retained  with  the  batches 
throughout  entry  processing. 

6.1.4.1    Professional  Scoring 

The  batches  of  student  booklets  were  sent  from  the  receipt  processing 
area  to  the  scoring  area  where  the  open-ended  reading,  science,  and 
mathematics  items  were  read  and  scored.     The  procedures  ancl  guidelines 
followed  in  scoring  these  items  are  more  fully  described  in  Chapter  6.2. 

Each  open-ended  item  was  provided  with  a  set  of  scannable  bubbles  to  be 
filled  in  by  the  reader.     The  bubbles  were  generally  at  the  bottom  of  the 
page  on  which  the  item  was  printed  to  avoid  distracting  or  confusing  the 
student.     When  several  open-ended  mathematics  were  printed  on  the  same  page, 
the  bubbles  were  printed  adjacent  to  each  item  to  facilitate  their  scoring. 
All  open-ended  reading  and  science  items  were  provided  with  an  extra  set  of 
bubbles  to  permit  secondary  scoring  of  the  primary  trait  scores  for  inter- 
rater  reliability  analysis.     Several  of  the  reading  items  that  were  to  be 
evaluated  for  secondary  traits  had  an  additional  set  of  bubbles  for  each 
secondary  trait  score. 

All  of  the  spiral  batches  and  some  of  the  bridge  batches  were  processed 
by  four  readers:  the  primary  and  secondary  reading  and  science  reader,  and 
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the  primary  and  secondary  mathematics  reader.    The  mathematics  read*>rs  were 
located  in  a  separate  area  from  the  reading  and  science  readers.     Each  area 
had  three  sets  of  shelves  for  controlling  the  processing  of  the  booklets. 
The  first  shelf  held  the  batches  to  be  scored  by  the  primary  reader  the 
second  had  the  batches  for  the  secondary  reader,  and  the  third  for  the 
completed  batches  to  be  forwarded  to  the  next  area. 

The  primary  reading  and  science  reader  would  examine  each  booklet  in  a 
batch  and  determine  if  it  contained  any  reading  or  science  open-ended  items. 
It  so,  the  reader's  identification  code  was  written  in  and  gridded  in  the 
second  column  of  bubbles  in  the  reader  identification  area  on  the  inside 
front  page.     The  reader  would  then  locate  and  read  each  of  the  open-ended 
items  for  that  booklet  and  grid  the  first  primary  trait  score  and  all 
secondary  trait  scores  into  the  appropriate  bubbles.     For  every  fifth  booklet 
read    the  reader  would  place  a  piece  of  tape  over  the  primary  trait  score 
bubbles  in  order  not  to  influence  the  secondary  reader.     The  completed 
booklets  were  stacked  in  the  same  order  in  which  they  were  received  and  the 
completed  batch  was  placed  on  the  second  shelf. 

The  secondary  reading  and  science  reader  s^:v,cted  every  fifth  eligible 
booklet  from  the  batch  in  order  to  achieve  a  20  percent  rate  of  reliability 
scoring.     This  reader's  identification  code  was  entered  into  the  third  column 
of  the  reader  identification  area.     The  reader  then  located  the  items  with 
concealed  primary  trait  scores,  read  and  scored  them,  and  removed  the  pieces 
of  tape.     The. completed  batch  was  placed  on  the  third  shelf,  to  be  forwarded 
to  the  mathematics  scoring  area. 

The  primary  mathematics  reader  examined  each  booklet  for  the  presence  of 
open-ended  mathematics  items.     These  booklets  were  then  gridded  with  the 
reader's  identification  code  in  the  first  column  of  bubbles.     The  responses 
were  then  evaluated  for  correctness  and  the  scores  gridded  into  the 
appropriate  bubbles.     The  completed  batch  was  placed  on  the  second  shelf  for 
processing  by  the  secondary  reader.     This  reader  performed  a  correctness 
check  on  every  tenth  booklet  read  by  the  primary  reader.     If  this  reader 
disagreed  with  the  -core  given  by  the  primary  reader,  the  score  was  changed 
in  the  booklet  and  recorded  on  a  separate  roster.     The  completed  batch  was 
placed  on  the  third  shelf,  to  be  forwarded  to  scanning  processing 


6.1.4.2  Scanning 

Before  the  batches  of  scored  booklets  could  be  sent  to  scanning  they 
had  to  be  grouped  by  age  cohort  and  placed  into  "capsules"  that  were 'then 
arranged  sequentially  on  "carts."    The  capsules  were  cardboard  boxes  with  one 
open  cide  to  facilitate  access  by  scanning  and  resolution  staff,  and  hangers 
on  the  other  side  to  permit  removal  from  the  carts.    The  carts  were 
transportable,  two-sided  hanging  shelves  with  sloping  sides  to  permit  the 
capsules  to  hang  with  the  open  sides  out  while  keeping  the  documents  in.  The 
carts  were  shipped  to  the  scanning  area. 

The  first  step  in  the  scanning  process  was  to  separate  each  booklet  into 
Its  component  pages  for  single -sheet  processing  by  the  scanner.     Each  booklet 
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was  secured  by  three  staples  along  the  left  edge.     The  timing  marks  for  the 
scanner  were  also  printed  along  this  edge.     Two  special  machines  were  used  to 
cut  off  the  stapled  edge  without  damaging  the  timing  marks.     The  guillotine 
could  cut  three  or  four  booklets  at  a  time  but  required  a  slower,  manual 
setup  process.     The  slitting  machine  was  more  automatic,  processing  one 
booklet  at  a  time,  but  was  less  precise  than  the  guillotine.  Careful 
handling  of  these  booklets  was  imperative  once  they  were  cut,  as  the  scanning 
program  depended  on  the  correct  sequencing  of  pages  within  each  booklet.  The 
guillotined  booklets  were  placed  back  in  their  capsules  and  the  completed 
cart  sent  to  the  scanning  machine. 

The  scanning  machine  operator  identifier*  'he  age  cohort  of  the  booklets 
to  be  scanned  and  started  up  the  appropriate  program  on  the  computer.  A 
magnetic  tape  was  pulled  from  the  scratch  pile  and  mounted  on  the  tape  drive. 
Scanning  was  initiated  by  placing  the  sheets  from  the  first  capsule  into  the 
input  hopper  of  the  scanni.ig  device.     The  scanner  then  read  both  sides  of 
each  sheet  and  placed  it  into  one  of  two  hoppers.     If  no  errors  in 
readability  or  sequencing  were  detected,  the  sheet  went  into  the  output 
hopper  and  the  next  sheet  was  read  from  the  input  hopper.     If  an  error  was 
indicated,  the  sheet  was  diverted  into  the  shunt  hopper,  the  program  wrote  an 
informational  message  to  the  operator's  console,  and  the  scanner  stopped 
processing  while  the  operator  took  appropriate  action. 

Each  page  of  every  booklet  was  printed  with  a  set  of  identification 
marks  next  to  the  timing  marks.     The  front  cover  of  each  booklet  number  had  a 
unique  set  of  these  marks,  and  the  pages  within  each  block  type  were 
similarly  identified  by  block  code  and  sequence  number.    As  the  scanner  read 
a  booklet  cover,  the  program  identified  the  booklet  number  and  referred  to  an 
internal  table  to  determine  which  blocks  were  to  follow  and  which  page 
formats  were  within  each  -.-lock. 

If  a  page  sequence  error  was  indicated,  the  operator  instructed  the 
program  to  treat  the  page  as  missing  and  placed  the  shunted  page  into  the 
input  hopper  to  be  read  again.     If  a  page  within  a  block  was  unreadable,  che 
operator  again  instructed  the  program  to  treat  it  as  missing  and  placed  the 
sheet  perpendicularly  on  top  of  the  output  stack.     If  a  block  sequence  error 
or  unreadable  booklet  covei  was  indicated,  the  operator  instructed  the 
program  to  insert  a  dummy  record  and  removed  the  remaining  pages  of  that 
booklet  and  placed  them  perpendicularly  on  top  of  the  output  stack. 

As  each  batch  completed  scanning  processing,  it  was  removed  from  the 
output  hopper  and  placed  back  in  its  capsule.     The  next  batch  was  taken  from 
its  capsule  and  placed  into  the  input  hopper  and  the  machine  started  again. 
When  the  last  batch  was  completed,  the  operator  terminated  the  prograa, 
dismounted  the  tape,  and  removed  the  listings  from  the  printer. 

The  output  data  tapes  were  forwarded  to  the  VAX  computer  area  for 
loading  processing.     The  scanned  documents  were  returned  in  their  original 
cartons  to  the  resolution  processing  area. 
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6.1.4.3  Loading 


The  scanning  tapes  were  received  and  checked  in  by  an  operator  at  the 
VAX  computer  area.     The  operator  initiated  the  loading  progrL  by  selecting 
the  second  option  on  the  data  entry  menu.  6  f    6        oy  selecting 

_    The  program's  first  input  request  was  the  tape  number,  a  six-dieit  code 
printed  on  an  external  label  on  the  tape  and  coded  internally  by  the  scanning 
program.     The  operator  then  mounted  that  tape  on  the  tape  drlve'^and  put  the  ^ 

proleedei  II"  '^"'^"^  ^^^^^  ^^P^  been  mounLd  and 

nrint!H  ^k''  ?  ^^^'"^  *  processed  the  tape,  the  program 

printed  the  batch  code  and  record  count  for  each  batch  to  the  operator's 
terminal,   to  assure  the  operator  that  the  program  was  running  properly.  When 
the  program  reached  the  end  of  the  tape  file,   it  printed  out  three  listings 

listings  consisted  of  an  error  log,  a  batch  listing,  and  an  audit  listing. 

J^"  ^""r^^?^  "'^^  ^  running  commentary  and  summary  of  the  processing  of 
lTt:lTlaJ  identified  with  the  tape  number,  file  name,  and  fate 

of  the  loading  run.  The  start  of  each  batch  was  recorded  with  the  batch 
nun.ber  and  its  corresponding  school  and  session  codes.  Any  disagreement 
between  these  codes  and  those  entered  from  the  school  worksheet  was  recorded 

h'-^      K^'^if  •  b-l°-S  ^°  —ion  type  (e  g 

bridge  booklets  m  a  spiral  session)  were  also  listed  here  as  well  as  all 
unscannable  booklets.     At  the  end  of  each  batch,   the  number  of  scannable  and 
unscannable  booklets  were  printed.  ^^^nuidoxe  ana 

of  reported  the  information  from  the  front  cover  fields 

iLZt.    I          T""/":  ^^'^  ^^'^  ^^^^^"S  could  be  checked  against  the 

administration  schedules  for  discrepant  or  missing  information. 

K.oh  ^""^^^  1^"^''"''^  identified  the  data  problems  found  within  each  batch 

Each  data  anomaly  was  identified  by  the  batch  sequence  number,  booklet 
number,  section,  and  item  number  to  facilitate  location  of  the  data  in  the 
raw  instruments  by  resolution  staff. 

-.^  '^u"  printed  output  was  forwarded  to  the  resolution  area  to  be  joined 
with  the  scanned  materials.     The  tape  was  retained  in  the  VAX  computer  area. 

6.1.4.4    Resolution  Processing 

The  error  log  and  batch  listing  were  retained  by  the  operations 
coordinator.     The  audit  listings  were  separated  by  batch  nu;ber  and  matched 
with  the  appropriate  scanned  materials.     If  the  error  log  indicated  any 
unscannable  booklets  within  a  batch,  they  were  pulled  and  manually  entered 
and  verified  through  the  data  entry  system.     Upon  completion  of  verification 
processing,  the  system  produced  an  updated  audit  listing  that  replaced  the 
one  output  from  the  load  process. 

Staff  assigned  to  resolution  processing  reviewed  the  audit  listing 
checked  the  actual  responses  in  the  booklets  wherever  asterisks  or  question 
marks  were  indicated,  determined  the  appropriate  value(s)  to  be  coded  in  the 

III 


data  file,  and  wrote  these  new  codes  on  the  audit  listing.  The  asterisks 
indicated  multiple  gridding  of  a  single-response  item;  question  marks  flagged 
critical  fields  from  the  front  cover,  such  as  sex  or  birthdate,  that  were 
incorrectly  gridded  or  out-of -range  and  fields  from  unscannable  pages. 

Access  to  the  student  data  for  entry,  verification,  or  resolution 
processing  was  gained  through  the  third  option  on  the  data  entry  menu.  The 
first  screen  (Figure  6.1.4)  requested  the  identification  number  of  the  batch 
to  be  processed;  the  PSU,  school,  and  session  codes  as  a  secondary  check  on 
the  batch;  and  a  code  for  the  processing  mode.     The  second  entry  screen 
(Figure  6.1.5)  prompted  for  input  of  the  batch  serial  number  and  the  student 
ID  number  as  a  secondary  check. 

If  the  program  was  in  the  entry  mode  and  no  data  record  for  the  booklet 
could  be  found,  the  program  would  set  up  to  create  a  new  record  and  request 
entry  of  the  booklet  cover  data.     If  in  verification  mode  and  the  data  record 
had  not  been  already  verified,  the  program  would  request  re-entry  of  the 
cover  data  and  compare  against  the  data  record.     If  in  resolution  mode  and 
the  data  record  had  been  through  verification  or  loading  processing,  all  data 
fields  were  displayed  and  the  operator  could  either  modify  these  fields  or 
advance  to  the  rest  of  the  entry  screens  for  that  booklet. 


Figure  6.1.4 


Student  Session  Data  Entry  Screen 


NAEP  YEAR  17  STUDENT  DATA 


BATCH: 


PSU: 
SCHOOL: 


SESSION: 


MODE: 
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Figure  6.1.5 
Student  Booklet  Cover  Data  Entry  Screen 


STUDENT  ASSESSMENT  BOOK 

BATCH  SERIAL    STUDENT  ID:  

BOOK  NO.   

.     EA:   G:_     SEX:^  YEAR  17 

AGE  CLASS  _ 
BLOCKS 


R: 


TEACHER  INFORMATION 

NOW  EVER  NEVER 

1  _  _  _ 

2  _  _  _ 

3  _ 

4  _  _  _ 
5 


SCORERS 
SI:         S2:  S3: 


The  resolution  mode  of  the  entry  system  permitted  the  operator  to  access 
data  records,  display  the  field  values,  and  make  corrections  to  individual 
fields.     A  change  in  any  data  field  under  resolution  mode  also  generated  a 
record  for  the  audit  file,  «fhd  the  program  produced  an  updated  audit  listing 
at  the  completion  of  resolution  processing  for  each  batch.     There  was  no 
limit  to  the  number  of  times  a  session  or  data  record  could  be  processed 
under  resolution. 

On  completion  of  resolution  processing,  each  bundle  was  stored  in  a 
labeled  box  and  held  for  final  editing  and  quality  control  processing. 

A  final  validation  was  performed  when  the  data  entry  work  files  were 
spooled  onto  a  master  student  data  file.     This  spooling  program  checked  every 
data  field  of  every  student  record  for  out-of-range  values  and  question 
marks.     A  listing  similar  to  the  audit  listings  for  each  session  was 
produced,  which  resolution  staff  then  used  to  identify  and  correct  the 
remaining  data  anomalies. 

The  quality  control  process  selected  a  random  sample  of  each  booklet 
type  from  the  master  student  file,   identifying  those  booklets  for  extraction 
from  the  raw  data.     The  designated  booklets  were  located,  pulled  from  their 


-109- 


boxes,  and  forwarded  to  quality  control  staff.     The  responses  in  each  booklet 
were  then  compared  with  their  coded  data  values  in  the  data  file.     The  full 
details  and  results  of  the  quality  control  process  are  presented  in  Chapter 
6.5.     On  completion  of  quality  control  processing,  the  booklets  were  returned 
to  their  boxes. 

When  open-ended  items  in  an  assessment  also  have  been  used  and 
professionally  scored  in  a  previous  assessment,  a  reliability  check  of  the 
current  scorers  versus  the  previous  scorers  is  required.     To  accomplish  this 
task  it  was  necessary  to  identify  booklets  that  contained  a  block  of  items 
within  the  given  subject  area,  extract  the  booklets  from  the  spiral  batches, 
and  store  them  for  future  use.     A  computer  program  was  developed  to  generate 
listings  by  age  level  of  the  booklets  to  be  extracted.     The  listings 
contained  the  batch  number,  sequence  number  of  the  booklet  within  a  batch, 
and  the  student  booklet  number.     Operations  staff  went  through  each  batch 
extracting  the  appropriate  uooklets  and  boxing  them  by  subject  area  booklet 
type.     These  booklets  were  then  shipped  to  the  ETS  data  retention  area  for 
future  use.     At  the  completion  of  the  operational  process  all  remaining 
assessment  booklets  were  shipped  to  the  ETS  data  retention  area  for  long-term 
storage . 


6.1.5  QUESTIONNAIRES 

The  questionnaire  instruments  were  separated  by  type  and  accumulated  by 
the  operations  coordinator  as  they  were  received  from  mail  processing.  These 
data  were  also  transcribed  through  the  NAEP  data  entry  system  but  on  a  lower 
priority  basis  than  the  student  booklets.     The  excluded  student 
questionnaires  received  higher  priority  than  the  teacher  and  school 
questionnaires,  since  the  demographics  of  the  excluded  students  were  used  in 
deriving  the  sampling  weights  of  the  assessed  students.     Every  effort  was 
made  to  keep  the  processing  rate  of  these  instruments  in  pace  with  the 
student  data  entry,  in  order  to  have  the  two  files  completed  at  the  same 
time. 

Processing  of  the  questionnaire  data  was  initiated  by  selecting  the 
fourth  option  on  the  Data  Entry  menu.     The  first  entry  screen  (Figure  6.1.6) 
prompted  for  input  of  the  questionnaire  type,  age  group,  and  processing  mode. 
The  questionnaire  entry  programs  followed  the  same  model  as  the  student  entry 
program  with  the  absence  of  a  tracking  file  and  session  batching.  Entry, 
verification,  and  resolution  modes  were  available;  audit  reports  were 
initiated  by  the  operations  coordinator. 


ERLC 


-no- 

114 


Figure  6.1.6 
Questionnaire  Data  Entry  Screen 


NAEP  YEAR 

17  QUESTIONNAIRE  MENU 

TYPE: 

AGE: 

MODE: 

1 

SCHOOL  1 

AGE  9 

1 

ENTRY 

2 

TEACHER  2 

AGE  13 

2 

VERIFICATION 

3 

EXCLUDED  STUDENT  3 

AGE  17 

3 

RESOLUTION 

4 

COMPUTER  COORDINATOR 

The  excluded  student  questionnaire  entry  program  first  displayed  a 
screen  for  entry  of  the  front  cover  data.     The  operator  was  prompted  for  the 
serial  numoer  of  the  booklet  to  be  processed.     An  error  condition  occurred  if 
either  a  record  with  that  serial  number  was  found  under  entry  mode  or  no 
record  was  found  under  verification  or  resolution  mode.     In  either  case  the 
operator  was  asked  to  verify  that  the  correct  number  had  been  entered  If 
the  problem  persisted,  it  was  referred  to  the  operations  coordinator  for 
resolution.    The  remaining  cover  information,  including  PSU  and  school  code 
student  sex,  ethnicity,  grade,  and  birthdate,  were  processed  as  for  the 
student  booklet  covers.     The  program  then  displayed  a  single  screen  for 
L^^^r^^"^         responses  within  the  questionnaire.     When  the  operator  pressed 
ENTER  to  terminate  processing  for  that  booklet,  the  program  redisplayed  the 
cover  entry  screen,  ready  to  process  another  booklet.     A  blank  field  entered 
in  the  serial  number  field  returned  the  program  to  the  primary  menu. 

The  teacher  questionnaire  entry  program  first  displayed  a  screen  for 
ertry  of  the  cover  information.     It  processed  the  serial  number  in  the  same 
t£.shion  as  did  the  excluded  student  questionnaire  entry  program.     The  cover 
iTiformation  only  included  the  PSU,  school,  and  teacher  codes.     As  the  longest 
questionnaire  instr^iment,   the  teacher  questionnaire  required  three  screens 
for  entry  processing  due  to  software  limitations  as  well  as  general 
appearance  and  ease  of  reading.     Completion  of  processing  for  each  booklet 
returned  the  program  to  che  cover  entry  screen,  where  the  entry  of  a  blank 
serial  number  returned  the  program  to  the  primary  mfmu. 

The  school  questionnaire  entry  program  also  started  with  a  display  of 
the  cover  entry  screen.     The  only  information  requested  for  this  instrument, 
however,  was  the  PSU  and  school  code,  which  also  served  as  the  booklet 
Identification  number.     Due  to  the  large  number  of  questions  in  this 
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questionnaire,  entry  processing  required  two  screens.     Completion  of 
processing  for  each  booklet  returned  the  program  to  the  cover  entry  screen, 
where  the  entry  of  a  blank  PSU  and  school  code  returned  the  program  to  the 
primary  menu. 

After  all  questionnaires  had  been  received  and  processed  through  the 
entry  system,  a  validation  program  was  run  against  all  data  values  in  all 
records.    All  remaining  data  errors  or  discrepancies  were  then  corrected 
using  the  resolution  mode  of  the  entry  system.    A  final  audit  listing  was 
generated,  recording  all  entry  activities  for  each  questionnaire. 

The  questionnaires  were  subjected  to  the  same  quality  control  procedures 
that  the  student  data  received.     The  details  of  the  sampling  rates  and 
results  are  discussed  in  sections  6.5.2  through  6.5.4. 

At  the  completion  of  quality  control  processing,  the  questionnaires  were 
packed  into  boxes  and  shipped  to  the  ETS  data  retention  area  for  long-term 
storage . 
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Chapter  6.2 
PROFESSIONAL  SCORING 


Anne  Campbell 
Educational  Testing  Service 


The  professional  scoring  of  the  1986  NAEP  assessment  was  conducted  for 
open-ended  reading,  mathematics,  science,   -nd  computer  competence  items  from 
all  three  grade/ages.     Three  different  groups  of  scorers  were  assigned  to  do 
the  scoring:     one  group  for  mathematics,  another  group  for  reading  and 
science,  and  a  third  group  for  computer  competence.     The  two  groups  for 
mathematics  and  reading/science  worked  concurrently;  a  separate  scoring 
operation  was  conducted  later  to  score  the  open-ended  computer  competence 
items . 

The  1986  NAEP  assessment  included  213  open-ended  mathematics  items:  37 
at  grade  3/age  9,  86  at  grade  7/age  13,  and  90  at  grade  11/age  17.     Of  those 
items,   13  were  used  at  both  grade  3/age  9  and  grade  7/age  13;  69  were  used  at 
both  grade  7/age  13  and  grade  11/age  17;  and  6  were  used  at  all  three 
grade/ages.     Also  included  in  the  assessment  were  27  open-ended  reading, 
science,  and  computer  competence  items  as  listed  in  Table  6.2.1.     This  table 
provides  an  overview  of  the  items,  including  NAEP  number,  grade/age  level, 
and  score  ranges . 

The  rest  of  this  chapter  will  include  a  description  of  the  scoring 
schemes  and  will  discuss  the  scoring  operation,  including  training,  work 
flow,  and  reliability. 


6.2.1    DESCRIPTION  OF  SCORING 
6.2.1.1  Mathematics 

All  open-ended  mathematics  items  were  scored  on  a  right-wrong  basis: 
1-correct;  2-incorrect  (omitted  responses  were  scored  as  0).    Answers  written 
on  the  answer  lines  were  the  basis  for  the  scores;  however,  if  the  student 
left  the  answer  line  blank,  consideration  was  given  to  answers  written  under 
the  problem  or  answers  written  where  the  student  had  figured  out  the  problem. 
Scores  were  indicated  by  gridding  in  scoring  ovals  at  the* bottom  of  the  page 
where  each  item  appeared. 


6.2.1.2  Reading 

All  opeT-ended  cognitive  reading  items  were  scored  according  to  criteria 
developed  for  each  item.     These  criteria  were  defined  to  evaluate  how  well 
students  responded  to  a  reading  passage  when  asked  to  perform  such  tasks  as 
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Table  6.2.1 


Distribution  of  Open-Ended  Reading,  Science 
and  Computer  Competence  Items 


NAEP  Item 


Reading  (R) 
Science  (S) 


Grade/Age 


Primary 
Score 


Item  Name 

Number 

Computer(C) 

3/9 

"7/10 

11    /I  "7 

11/17 

Ranges 

Inside  My  Head 

bUU /UUi 

K 

X 

X 

X 

U-  9 

Eggplant  I 

Kin  0 1  Q  m 

•n 
K 

X 

V 

X 

Jiggpiant  ii 

XT  r\  o  1  one 

R 

X 

X 

0-4,  9 

Goods  to  Market 

ry 
K 

V 
A 

X 

X 

O     C     "7     O  O 

Jacob 

0 

K 

X 

X 

X 

O    A  O 

u-4 ,  y 

ua  L.  ue  ty/  duxd 

VTA  1  '5  A  m 

o 

V 

X 

V 

X 

u  -  J ,  y 

L»anoie 

M/ii  A7m 

INM-lO  /  U  1 

r 

V 

X 

n  o  o 

u  -  z ,  y 

L>  irCULC 

MA  0 A  ono 

c 

X 

U  -  J  ,  y 

Hours  of  Daylight 

MA  0  n "/  m 
NhZU /Ui 

c 
b 

X 

0-3,9 

Liquids  Freeze 

MA  0 "7  m 
Nh-Z  J  /Ui 

b 

X 

u-4 ,  y 

Pendulum 

MA  '5  nft  m 

b 

X 

u-  J ,  y 

riant  L»eii 

J  i JUi 

b 

X 

0-4,9 

o  a  JL  u  /  o  anci 

MA  0  '3  0  m 

c 

b 

V 

X 

n  A  o 
U  -4 ,  y 

onaKe/n.ou.>e 

MA  0  Q  om 

b 

X 

n   A  o 
U  -4 ,  y 

c 

Y 

2  Batteries/Bulb 

N437001 

s 

X 

0-4,9 

Cats  and  Dogs 

N605301 

c  • 

X 

X 

X 

0-3,9 

Computer  5  Times 

N605701 

c 

X 

X 

0-3,9 

Logo  Pictures 

N609201 

c 

X 

X 

0-4,9 

Castle/Raccoon 

N609602 

c 

X 

0-3,9 

Jobs  1 

S603401- 

•2 

c 

X 

0-9,99 

Jobs  2 

S603501- 

-2 

c 

X 

0-9,99 

Quiz  1 

N606101 

c 

X 

0-3,9 

Quiz  2 

N606102 

c 

X 

0,1.3,9 

Quiz  3 

N606103 

c 

X 

0-3,9 

Basic  Program 

N608401 

c 

X 

0-3,9 

Ace  Computer 

S604501 

c 

X 

0-3,9 

Secondary 
Score 
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Identifying  the  author's  message  and  substantiating  their  interpretation 
predictmg  on  the  basis  of  the  reading  passage,  supporting  an  interpretation 
and  comparing  and  contrasting.     Criteria  for  each  item  were  associated  with  ' 
specific  score  points  in  a  scoring  guide.     The  guides  included  score  points 
of  0  to  4  or  0  to  5  and  9  (and  in  one  case  score  points  of  7  and  8).  Readers 
assigned  scores  of  0  and  9  (and  for  the  one  exception,  scores  of  7  and  8)  to 
ITlZT.f    t\lT  Jl^-l^'  '^^d^-ipherable,  off-task,  or  contained  a  statement 
to  the  effect  that  the  student  did  not  know  how  to  do  the  task  The 
remaining  scores  defined  a  continuum  of  success  in  completing  the  task,  with 
a  score  of  1  indicating  an  unsatisfactory  response  and  a  score  of  4  or  5 
indicating  a  response  that  went  beyond  the  essentials  by  providing  more 
detail  and  being  more  coherent.     (Note  that  because  of  .changes  in  the  way 
that  scoring  standards  were  applied,  the  1986  results  on  item  N003100,  "Goods 
to  Market,     are  not  strictly  comparable  to  the  1984  results.)     Some  items 
also  required  secondary  scores,  which  generally  involved  categorizing  the 
kind  of  evidence  or  details  the  student  used  as  support  for  an 

^^'"•^''^  ^^^^  secondary  scores  were  gridded  in  scoring  ovals 
at  the  bottom  of  the  page  where  each  item  occurred.     One  reading  background 
Item  was  scored  according  to  a  guide  in  which  the  score  points  categorized 
the  type  of  response  the  student  gave. 

6.2.1.3  Science 

The  open-ended  science  items  were  also  scored  according  to  a  rubric 
developed  for  each  item.     The  criteria  for  each  score  point  focused  on  how 
correctly  the  student  answered  the  questions.     With  the  exception  of  one  item 
So™  Tl  s^o'^ed  on  a  correct-incorrect  basis,  the  scores  for  the  items  ranged 
from  0  to  3  or  0  to  4  and  9: 

0,9:  These  scores  were  given  to  responses  that  were  blank 

undecipherable,  off  task  or  contained  a  statement  to 'the 
effect  that  the  student  did  not  know  how  to  do  the  task. 

1:      This  score  indicated  an  incorrect  response  to  the 
question. 

2:      This  score  indicated  an  answ-r  that  was  correct  to  a 
point  but  contained  some  misinformation  or  was  too 
generalized. 

3:      This  score  indicated  a  correct  answer. 

4:      This  score,  when  present,  indicated  a  correct,  detailed 
answer . 

Item  scores  were  gridded  in  scoring  ovals  at  the  bottom  of  the  page 
where  each  item  occurred. 
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6.2.1.4    Computer  Competence 


The  scoring  of  the  open-ended  computer  competence  items  was  a  separate 
opera|:ion  from  the  scoring  of  the  other  items.    Most  open-ended  items  were 
scored  on  a  standard  scale: 

0  -  No  response 

1  -  An  attempt  was  made,  but  incorrect 

2  «  A  response  indicating  students  had  some  idea  of  what  they 

were  doing 

3  -  Correct  response 

4  Best  possible  response 

9  -  An  "I  don't  know,"  off- task,  or  irrelevant  response 

Some  (especially  noncognitive  items)  were  scored  by  grouping  responses  into 
various  categories . 

6.2.2    THE  SCORING  OPERATION 

6.2.2.1  Scorers 

Eight  persons  and  one  assistant  scoring  supervisor  who  also  scored 
reading  and  science  items  were  assigned  to  the  scoring  of  the  mathematics 
items  at  all  three  levels.     These  people  had  at  least  a  high  school 
education,  and  the  assistant  supervisor  was  an  experienced  mathematics 
teacher. 

Seven  persons  and  the  scoring  supervisor  from  the  NAEP  staff  scored  the 
open-ended  reading  and  science  items.     Generally  the  readers  had  at  least  BA 
degrees  in  science,  education,  or  English.     The  group  included  men  and  women 
of  various  ages  who  had  lived  in  various  parts  of  the  country.     One  of  the 
readers  with  a  strong  science  background  was  designated  as  the  science 
assistant  responsible  for  reviewing  discrepancies  with  science  items. 

Six  persons  and  a  NAEP  staff  member  scored  the  open-ended  computer 
competence  items.     A  majority  of  the  people  were  college  students  who  were 
majoring  in  technical  fields. 

6.2.2.2  Training:  Mathematics 

Because  the  scoring  for  the  mathematics  items  was  on  a  right-wrong- omit 
basis,  lengthy  training  was  not  necessary.     The  orientation  to  the  scoring 
involved  familiarizing  the  scorers  with  the  procedures  for  scoring 
mathematics  and  with  the  mathematics  guides,  which  consisted  of  a  listing  of 
the  right  answers  for  the  items  in  each  of  the  blocks. 
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6.2.2.3    Training:    Reading  and  Science 


Before  training,  the  NAEP  scoring  supervisor,  the  assistant  scoring 
supervisor,  and  the  science  assistant  worked  with  NAEP  test  development  staff 
to  prepare  training  sets  and  to  refine  the  scoring  guides. 

Training  was  done  cn  all  the  reading  and  science  items  at  all  three 
grade/age  levels.     Training  involved  explaining  the  item  and  its  scoring 
guide,  discussing  responses  that  were  representative  of  the  various  score 
points  in  the  guide,  and  then  scoring  and  discussing  approximately  65  to  100 
randomly  selected  responses  for  the  reading  items  and  25  to  50  responses  for 
the  science  items.     The  purpose  of  the  training  was  to  familiarize  the  group 
with  the  scoring  guides  and  to  reach  a  high  level  of  agreement  among  the 
scorers.     After  the  group  training  was  completed,  each  scorer  scored  the 
Items  in  each  of  nine  bundles  of  booklets.     Their  scores  were  recorded  and  a 
follow-up  session  was  held  to  discuss  those  responses  for  which  there  was  a 
wide  range  of  scores.     Once  the  follow-up  session  was  completed,  the  scoring 
began.     Initial  training  was  completed  in  approximately  one  week. 

As  a  follow-up  to  training,  notes  on  various  items  were  compiled  and 
distributed  to  the  readers  for  their  reference.     In  addition,  short  training 
sessions  were  conducted  when  the  scoring  supervisor  ascertained  in  reviewing 
discrepancies  that  particular  items  were  presenting  difficulties.  The 
scoring  supervisor  also  consulted  with  individual  readers  as  the  scoring 
progressed.    When  a  reader  was  judged  to  be  causing  a  discrepancy,  the 
supervisor  would  discuss  the  response  and  its  score  with  that  reader. 


6.2.2.4    Training:     Computer  Competence 

Training  for  the  computer  competence  items  followed  the  same  procedures 
as  for  reading  and  science.     Short  training  sessions  were  also  held  whenever 
peculiar  responses  occurred  that  had  not  been  covered  in  the  initial 
training.     The  initial  training  lasted  for  one  and  one-half  days. 


6.2.2.5    Assignment  of  Vork:    Mathematics,  Reading,  and  Science 

Two  separate  groups  of  scorers  were  scoring  simultaneously,  one  group 
scoring  the  mathematics  items  and  the  other  group  scoring  the  reading  and 
science  items.     Batches  of  booklets  were  first  scored  by  the  reading/science 
readers  and  then  sent  on  to  the  mathematics  scorers.     Both  the  mathematics 
scorers  and  the  reading/science  readers  received  the  booklets  in  batches  as 
they  were  received  from  the  schools.    A  particular  scorer  scored  either  all 
the  booklets  in  a  bundle  that  had  mathematics  items  or  all  the  booklets  that 
had  reading  and  science  items.     Because  of  the  spiral  design,  a  scorer  would 
encounter  many,   if  not  all,  of  the  items  at  a  grade/age  level  as  he  or  she 
scored  a  batch  of  booklets.     Furthermore,  the  scorers  cycled  themselves 
through  the  three  grade/age  levels  so  that  the  scorers  had  continual  exposure 
to  all  items  at  all  ages  throughout  the  scoring. 
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The  1986  assessment  included  one  open-ended  reading  item  that  had  also 
been  administered  in  1984.     To  ensure  consistent  scoring  across  assessments, 
the  1986  readers  were  trained  on  sample  responses  from  both  the  1984  and  the 
1986  assessments.     Then,  to  check  for  reliability,  a  20  percent  subsample  of 
the  1984  responses  was  retrieved,  the  scores  were  masked,  and  the  responses 
were  distributed  to  and  rescored  by  the  readers.     This  rescoring  was 
performed  concurrently  with  the  scoring  of  the  1986  responses. 

6.2.2.6  Assignment  of  Work:     Computer  Competence 

For  the  scoring  of  the  computer  competence  items,  the  booklets  with 
those  items  were  sorted  by  booklet  number  and  then  batched  in  bundles  of  25. 
A  scorer  then  scored  all  the  items  in  all  the  booklets  in  a  particular 
bundle.     Scorers  also  cycled  themselves  through  the  booklets  at  the  various 
grade/age  levels  to  ensure  consistency  in  scoring  overlap  items.  Those 
scorers  with  minimal  computer  programming  background,  however,  did  not  score 
the  programming  items  at  grade  7/age  13  and  grade  11/age  17. 

6.2.2.7  Reliability  and  Resolution 

Ten  percent  of  the  mathematics  items  were  subject  to  a  correctness 
check,   in  which  a  second  scorer  checked  to  see  that  the  first  scorer  had 
correctly  scored  the  items.     If  the  second  scorer  found  a  mistake  in  scoring, 
he  or  she  corrected  it.     To  keep  track  of  the  reliability  of  each  scorer,  the 
second  scorer  kept  a  tally  of  how  many  items  he  or  she  checked  each  of  the 
other  scorers  and  a  tally  of  how  many  times  he  or  she  had  to  correct  a  score. 
This  procedure  was  followed  because  the  mathematics  scoring  was  done  on  a 
right -wrong- omit  basis  and  because  the  scoring  guides  were  exact  as  to  the 
correct  answers.     Results  of  this  correctness  check  showed  that  the  first 
scorer  was  correct  99  percent  of  the  time. 

Twenty  percent  of  the  reading  and  science  items  were  subject  to  a 
reliability  check,  which  entailed  a  second  reading  by  a  different  reader.  To 
prevent  a  second  reader  from  being  influenced  by  the  first  reader's  scores, 
the  first  reader  masked  all  the  scores  in  every  fifth  booklet  in  a  batch. 
These  booklets  were  passed  along  to  a  second  reader.     All  discrepancies  were 
then  reviewed  by  the  scoring  supervisor  or  science  assistant. 

The  same  general  procedures  were  followed  for  the  computer  competence 
scoring:     twenty  percent  of  the  responses  were  rescored;  second  scorers  did 
not  see  the  first  scores;  discrepancies  were  checked  by  the  scoring 
supervisor. 

In  analyzing  reader  reliability,  two  statistics  were  chosen:  the 
percent  of  exact  agreement  and  the  reliability  coefficient.     The  percent  of 
exact  agreement  is  the  percentage  of  times  that  the  two  readers  agreed 
exactly  in  their  ratings.     The  reliability  coefficient  is  the  intra-class 
correlation  among  readers.     The  results  for  each  grade/age  are  shown  in  Table 
6.2.2.     The  number  of  responses  analyzed  is  indicated  first;  the  second 


-120- 


CO  u 

Q)  O 

4J 
C 
<U 
O 
U 
d) 


•  : 
• 

(0 

I-I  0) 

<U 

<u 

<U  U 

•H 

O 

•H 

CO 

O  PM 
1 

^-^  s 

1 

o  u 

1 

O  M 

•  ^\ 

•H  c3 

<U  <l) 

cd  4-) 

'  -J 

1— 1  <D 

i-i  a 

1 

V-l  o 

o  o 

o 

0) 

0)  0) 

CO  <!) 

CO  4-> 

qj  ^ 

u  <: 

1— J  Cl> 

o 

O  S 

1  PM 

(fl  o 

1 

i-i  o 

» 

4-) 

Cvl 

• 

CM 

• 

VO 

o 

'  *i 

'  '"'1 

d) 

1  0)1 

1— 1 

1 

»0 

S  o 

<1)  c/^ 

CO 

r  1 

i-i 

<u  <u 

bO  bO 

tj  u 

J-i  <i 

a 

•H  (D 

O  CcJ 

'  131 

0)  0) 

o  C 

O  W 

CO  I 

V  0) 

o  a* 

cti  o 

00 

O  iH 

o 

•H 

V 

p 

•H 

}-l 
O 
CO 
0) 

Q 

e 
<u 

4J 


a\  <f  m  00 
ov  a\  ON  ON 

0000 


00  I-I  <f 
0\  ON  o\  o\ 

0000 


m  <f 
I-I  VO  o 
m  m  CO  <f 


o\  oj  10  00  o 

0\  ON  0\  ON  o 


O  O  O  O  I-I 


00  CO  <J-  10  ON 
ON   ON   ON   ON  ON 


00000 


<t  O  00  o  <f 
ON  O  CO  00 
CO  CO  CO  <J-  10 


ON 
ON 


ON 
ON 


ON  ON  O 
ON    ON  O 


O  O  f-l 


ON  00  ON 

On  On  On 


000 


00  ON 
ON  ON 


O  O 


ON  ON 
ON  ON 


o  o 


On  IT)  m        00  CNJ 

CO  I-I  cNj  I-I 
<f  10  10      10  10 


00  ON 

ON  ON   ON  ON 

0000 


00  ON  00 

On  On  On  On 


0000 


VO  VO  CO 
On  ON  CO  I-I 
CO  CO  CO  CO 


ON 
ON 


ON 
ON 


00 

10 


ON 
ON 


ON 
ON 


CO 
O 
VO 


o  ON  ON 

On  On  O  On  ON 


O  O  f-l  O  O 


VO      ON  ON  ON 

ON  ON  On  On  ON 


00000 


00      I-I  10 
CO      ON  CM 
-cJ-  -cJ-  10  ^  10 


00 

On  On  ON 


000 


00  00 

ON  ON  ON 


000 


10  10 
O  CM 
<J-  <J-  CO 


VO      00  <j- 

ON  On  ON  On 


0000 


00  VO  CO 

ON  On  ON  ON 


0000 


m  ON  m  ^ 
CO  r-.  I-I 

CO  CM  I— I  CM 


00  CM  O  CO 
ON  ON  O  On 


O  O  f-l  O 


00  CO  00  CO 

ON   ON   ON  ON 


000 


CO  CO  CM  CM 
CO  CO  CO  CO 
<J-  <J-  <J-  <J- 


B 

<u 

VO 

u 

On 

•H 

0 

0 

On 

ID 

On 

CO 

Q* 

r>* 

-5 

VO 

<u 

CO 

4J 

U 

OJ 

(X 

0) 

I-I 

c 

0 

CO 

CO 

0) 

u 

«d 

I-I 

0 

4J 

o  >^ 

CO  .H 


CM 


ERLC 


column  is  the  percent  of  exact  agreement;  and  the  third  column  is  the 
reliability  coefficient. 

These  results  show  a  very  high  degree  of  agreement  between  readers. 
Table  6.2.3  summarizes  the  statistics  by  grade.     For  all  three  grades  no 
exercise  had  less  than  91  percent  exact  agreement;  several  items  had 
agreement  as  high  as  99  percent.     The  reliability  coefficients  are  also  high, 
ranging  from  .92  to  1.00. 


Table  6.2.3 

Reliability  Statistics  for  Scoring  of  Open-Ended  Items 


Number  of 
Grade  Exercises 

3  9 
7  14 
11  16 


Low  High 

Percent  Percent 

93.1  99.6 

93.3  99.5 

91.0  99.4 


Low  r  High  r 

.94  1 . 00 

.92  1.00 

.95  1.00 


6.2.2.8    Data  Entry 

When  the  professional  scoring  was  completed,   the  batches  of  booklets 
were  sent  to  scanning.     (See  Chapter  6.3  for  details  concerning  the  scanning 
process  and  Chapter  6.4  for  information  concerning  editing  data.) 
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CHAPTER  63 
Data  Transcription  Systems 
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Chapter  6.3 
DATA  TRANSCRIPTION  SYSTEMS 


Alfred  M.  Rogers 
Educational  Testing  Service 


The  transcription  of  the  student  response  data  into  machine -readable 
form  was  achieved  through  the  use  of  three  separate  systems:  scanning 
loading,  and  resolution. 

The  student  instruments  were  printed  in  scannable  format  to  allow  the 
transcription  of  marked  responses  in  the  booklets  to  computer- readable  form 
on  a  magnetic  tape  by  a  programmable  optical  scanning  machine.     The  first 
TJil        fl^  chapter  will  describe  the  scanning  equipment,   the  programs  and 
data  used  by  the  machinery,  and  the  ETS  quality  control  standards  and 
procedures . 

The  data  contained  on  the  scanning  tapes  were  edited  and  loaded  into  an 
online  data  entry  and  res9lution  system  similar  to  that  used  in  the  1984 
assessment.     This  loading  procedure  validated  each  scanned  data  field 

tT^  ''T''''^^  compatible  with  the  resolution  system,  and 

reported  all  problems  for  subsequent  resolution.     The  second  part  of  this 
chapter  details  the  loading  procedure. 

A  modified  form  of  the  data  entry  system  developed  for  the  1984 
assessment  was  used  for  lo solution  of  the  scanned  data,  entry  of  the 
documents  rejected  by  the  scanning  machine,  and  entry  of  the  questionnaire 
instruments.     The  third  part  of  this  chapter  will  provide  an  overview  of  this 
system,  which  is  fully  described  by  Rogers  (1987).  overview  ot  this 

Figure  6.3.1  is  a  schematic  diagram  that  represents  the  flow  of  student 
assessment  materials  through  the  data  transcription  system.     The  reader  may 
refer  to  this  diagram  for  clarification  of  the  relationships  among  the 
components  of  this  system. 


6.3.1  SCANNING 


The  student  booklets  were  scanned  on  a  National  Computer  System  W201 
scanning  system.     The  scanner  was  controlled  by  a  Hewlett  Packard  1000 
minicomputer.     This  system  also  included  a  disk  drive  for  storage  of  the 
scanning  programs,  a  tape  drive  for  the  output  of  scanned  data  records    and  a 
printer  for  the  periodic  dumping  of  records  for  quality  control  checking 
The  scanning  programs  used  were  specifically  written  for  NAEP  using  the 
assembler  language  of  the  Hewlett  Packard. 
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Figure  6,3.1 
Intelligent  Data  Entry  System 
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An  optical  scanner  operates  by  sweeping  a  horizontally  oscillating  light 
beam  across  a  vertically  moving  sheet  and  detecting  reflections  of  the  beam 
from  pencil  irarks.     The  hardware  logic  of  the  scanner  treats  the  page  as  a 
rectangular  array  of  scannable  areas,  each  of  which  is  assigned  a  reflectance 
value  from  0  to  15.     This  array  of  values  is  passed  to  the  scanning  program 
software,   to  be  translated  into  response  data. 

After  the  first  side  of  a  sheet  has  been  scanned,   it  is  pushed  through  a 
loop  that  brings  the  other  side  of  the  sheet  to  face  the  scanning  beam  A 
similar  array  of  reflectance  values  is  passed  to  the  program  that  must  then 
not  only  translate  it  into  data,  but  decide  whether  to  route  this  page  to  the 
output  hopper  and  read  in  the  next  sheet  or  route  it  to  the  shunt  hopper  and 
stop  processing. 

The  paper  and  inks  used  in  producing  scannable  documents  are  required  to 
have  very  low  reflectances .     A  special  set  of  marks  were  printed  down  one 
side  of  each  page  at  equally  spaced  intervals  to  enable  the  scanning  hardware 
to  align  each  sheet  and  adjust  the  scanning  rate  to  the  movement  of  the 
sheet.     These  timing  marks  were  printed  using  a  highly  reflective  ink. 

Each  page  of  each  item  block  had  its  own  unique  format  in  terms  of  the 
arrangement  of  the  response  and  scoring  bubbles.     The  scanning  program  had  to 
be  able  to  identify  a  given  page,  determine  which  parts  of  the  returned  array 
to  process,   interpret  the  reflectance  values,  and  transcribe  them  to  data 
codes  on  the  output  record.     Each  page  was  printed  with  a  set  of  marks  next 
to  the  timing  marks  that  were  used  by  the  program  to  uniquely  identify  it  by 
block  code  and  page  number.     The  booklet  covers  were  similarly  identified 
according  to  booklet  number. 

The  scanning  program  logic  used  two  sets  of  tables  to  control  scanning 
processing.     When  a  booklet  cover  was  scanned,  the  program  used  the  booklet 
number  and  the  first  table  to  determine  which  blocks  were  to  be  processed 
Each  block  code,  in  turn,  was  referred  to  the  second  table  to  determine  the 
number,   formats  and  sequence  of  its  constituent  pages.     By  reading  the 
booklet  cover,  the  program  "knew"  which  pageo  would  follow  and  in  what  order. 

The  scanning  program  rejected  pages  for  one  of  two  reasons:  unreadable 
or  out  of  sequence.     A  page  was  unreadable  if  the  timing  or  identification 
marks  had  been  corrupted  by  either  tearing,  improper  trimming,  or  confusing 
stray  pencil  marks.     If  the  unreadable  page  happened  to  be  a  booklet  cover 
th-  operator  would  instruct  the  scanner  to  pass  the  remaining  pages  up  to  the 
next  booklet  cover  into  the  shunt  hopper,  place  the  pages  perpendicularly  on 
top  of  the  output  stack,  and  resume  processing  with  the  next  booklet.  For 
any  other  page  type,  the  operator  instructed  the  program  to  substitute 
question  marks  for  the  data  values  for  that  page  and  proceed  with  the  next 
page. 

Pages  out  of  sequence  were  generally  attributable  to  collating  errors  in 
printing.     When  the  program  encountered  this  type  of  error,  the  operator 
would  direct  the  scanner  to  shunt  the  remaining  pages  of  the  booklet  and  then 
place  them  perpendicularly  on  the  output  stack. 


-127- 

'130 


The  scanning  program  wrote  three  types  of  data  records  onto  the  magnetic 
tape.     Th**.  first  was  a  batch  header  record,  containing  information  gridded 
onto  the  batch  header  sheet  by  receipt  processing  staff.     The  second  was  a 
data  record  containing  all  of  the  translated  marked  bubbles  from  all  pages 
within  a  booklet.     The  third  type  was  a  dummy  data  record,  serving  as  a  place 
holder  in  the  file  for  a  booklet  with  an  unreadable  cover  sheet.     The  origin 
code  was  a  data  field  written  in  the  same  location  on  all  records  to 
distinguish  them  by  type. 

The  batch  header  record  preceded  all  data  records  for  a  given  batch.  As 
the  scanning  program  processed  the  header  sheet,  it  retained  the  batch 
identification  code  and  initialized  a  batch  sequence  number  to  one.  The 
batch  identification  code  and  sequence  number  were  written  to  each  record; 
the  batch  header  record  always  received  a  sequence  number  of  one,  the  first 
data  record  was  assigned  number  two,  and  so  forth.     The  scanning  machine  was 
directed  to  stamp  the  batch  identification  code  and  sequence  number  on  each 
page  of  a  booklet.     This  greatly  facilitated  the  location  of  individual  pages 
within  batches  by  resolution  staff. 

Each  data  record  was  formed  by  collecting  the  transcribed  marked  bubbles 
from  each  page  of  a  booklet,  placing  them  into  a  buffer  area  within  the 
program,  and  writing  the  buffer  to  tape  when  the  last  page  of  the  booklet  had 
been  processed.     Several  options  were  considered  in  designing  the  format  of 
the  output  data  records.     One  which  required  a  fixed  column  position  for  each 
item  response  value  would  have  been  very  3 srge  due  to  the  number  of  items  in 
the  assessment,  and  very  sparse,  due  to  the  BIB  spiral  design.     Another  which 
had  the  response  data  strung  out  in  contiguous  fields  across  items  and  blocks 
was  more  consistent  with  the  format  of  the  data  records  in  the  NAEP  data 
entry  system,  but  would  have  been  difficult  to  check  in  listings  for  quality 
control.     The  format  adopted  for  this  assessment  had  fixed  column  positions 
for  the  booklet  cover  data  fields  and  scorer  identification  codes.  The 
response  data  started  at  fixed  positions  for  each  positional  block,  with  the 
item  responses  in  contiguous  fields. 

The  data  values  from  the  booklet  covers  and  scorer  identification  fields 
were  coded  as  numeric  data.     Unmarked  fields  were  coded  as  hyphens  (-)  except 
for  the  race,  sex,  grade,  and  birthdate  fields,  which  were  returned  as 
question  marks  (?)  to  alert  processing  staff  of  missing  or  uncoded  critical 
data.     Fields  that  had  multiple  marks  were  coded  as  asterisks  (*) ,     The  data 
values  for  the  item  responses  and  scores  were  returned  as  alphabetic  codes. 
The  multiple-choice,  single- response  format  items  were  assigned  codes 
depending  on  the  position  of  the  response  alternative;   that  is,   the  first 
choice  was  assigned  the  code  "A",  the  second  "B" ,  and  so  forth.     The  circle- 
all-  that  -apply  items  were  given  as  many  data  fields  as  response  alternatives; 
the  marked  choices  were  coded  as  "A"  and  the  unmarked  choices  as  hyphens. 
The  open-ended  items  had  10  bubbles  labeled  from  zero  to  nine;  a  marked  zero 
was  coded  as  "A",  a  marked  one  as  "B" ,  and  so  on  up  to  "J".     As  with  the 
cover  data  fields,  unmarked  responses  were  coded  as  hyphens  a;.d  multiple 
marks  as  asterisks.     The  fields  from  unreadable  pages  were  coded  as  question 
marks  again  as  a  flag  for  resolution  staff  to  correct. 
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6.3.2    DATA  LOADING  SUBSYSTEM 


Each  magnetic  tape  produced  by  the  scanning  system  contained  data  for 
one  or  more  assessment  sessions  for  one  of  the  age  groups.     The  data  records 
on  these  tapes  conformed  to  a  fixed  format.     This  data  now  had  to  be  edited 
for  type  and  range  of  response,  transformed  to  a  compressed  format  compatible 
with  the  data  entry  system  files,  and  loaded  into  the  database  for  resolution 
processing.    A  procedure  for  accomplishing  all  of  these  tasks  was  designed 
and  developed  for  this  assessment. 

The  data  records  on  the  scanning  output  tape  were  ordered  in  the  same 
sequence  as  the  paper  materials  were  processed  by  the  scanner.     A  record  for 
the  batch  header  preceded  all  data  records  belonging  to  that  batch;  each  set 
of  records  belonging  to  one  batch  were  separated  from  the  others  by  its  batch 
header  record.     The  origin  code  field  on  each  record  served  to  distinguish 
the  header  records  from  the  data  records. 

The  processing  of  each  batch  began  with  the  identification  of  the  header 
record.     The  batch  identification  number  on  this  header  record  provided  the 
link  between  the  subsequent  data  records  on  the  tape  and  the  tracking  file 
generated  by  the  school  worksheet  entry  program  in  the  data  entry  system. 
The  load  program  used  the  batch  identification  number  to  locate  and  retrieve 
the  processing  information  for  that  batch  from  the  tracking  file.  The 
program  then  verified  that  it  had  the  correct  batch  by  comparing  the  PSU 
school,  and  session  codes  gridded  on  the  header  record  with  the  same  codes  in 
the  tracking  record. 

If  a  batch  code  could  not  be  located  in  the  tracking  file,  the  program 
would  generate  a  new  tracking  record,  using  only  the  information  contained  on 
the  header  record,  and  record  this  condition  on  an  error  log  file.     If  a 
batch  code  could  be  located  but  the  school  or  session  codes  did  not  agree 
the  program  would  record  this  conflict  in  the  error  log  and  continue 
processing. 

The  batch  header  record  also  contained  the  date  that  the  session 
materials  were  batched  together,  and  the  number  of  booklets  batched  by  the 
receipt  processor.    This  information  was  transferred  to  the  tracking  record 
for  later  processing  and  reporting. 

The  reading  of  a  batch  header  record  also  initiated  the  generation  of 
two  new  files  in  the  entry  system  database:  the  data  file  and  the  audit  file 
As  the  program  processed  each  record  within  a  batch  from  the  tape  file,  it 
would  write  the  edited  and  reformatted  data  records  to  the  data  file  and 
record  all  errors  and  special  codes  in  the  audit  file.     The  data  fields  on  an 
audit  file  record  identified  each  data  problem  by  the  batch  sequence  number 
booklet  serial  number,  section  or  block  code,  field  name  or  item  number,  and 
data  value.     The  program  would  generate  a  listing  of  the  data  problems  after 
each  batch  had  been  processed,  to  be  printed  at  the  termination  of  the 
program. 
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As  the. program  processed  each  data  record  it  would  first  read  the 
booklet  number  and  check  it  against  the  batch  session  code  for  appropriate 
session  type  (bridge  or  spiral).    A  mis-match  was  recorded  in  the  error  log 
and  processing  continued.     The  booklet  number  was  then  compared  against  the 
first  two  digits  of  the  student  identification  number.     If  they  disagreed, 
due  to  improper  gridding,  a  message  was  written  to  the  error  log  and  the 
booklet  number  was  substituted  into  the  student  nximber.    The  remaining 
booklet  cover  fields  were  then  read  in  and  validated  for  range.     The  PSU  and 
school  codes  were  compared  against  those  from  the  tracking  record;  the  range 
of  grade  codes  was  dependent  on  the  age  cohort  being  processed;  and  the  range 
of  birth  dates  was  dependent  upon  the  session  type  as  well  as  the  age  cohort. 
All  data  values  that  were  out  of  range  were  replaced  with  question  marks  and 
recorded  on  the  audit  file.     All  data  fields  that  were  read  in  as  question 
marks  or  asterisks  were  also  recorded  in  the  audit  file.     The  booklet  cover 
data  fields  were  written  to  a  batch  listing  file  that  would  be  printed  at  the 
end  of  load  processing.     This  listing  could  be  compared  against  the 
administration  schedule  to  assist  in  resolving  booklet  cover  data  p  oblems. 

The  processing  of  the  scorer  identification  code  fields  was  not  as 
straightforward.     If  a  booklet  contained  any  open-ended  mathematics  items, 
the  first  scorer  field  should  have  been  filled.     If  it  contained  any  open- 
ended  reading  or  science  items,  the  second  field  should  have  been  filled  and 
the  third  may  have  had  data  if  it  had  been  part  of  the  reliability  sample. 
The  program  had  to  determine  from  the  booklet  number  whether  the  booklet 
contained  any  open-ended  items  and  of  what  type.     It  would  then  flag  as 
erroneous  any  incomplete  field  that  should  have  been  filled,  or  any  nonblank 
field  that  should  have  been  empty  and  record  the  error  in  the  audit  file. 
Further,   it  would  remember  how  many  scorer  fields  of  each  type  were  marked 
for  later  processing  of  the  open-ended  item  scores. 

The  edited  booklet  cover  and  scorer  identification  fields  were  appended 
to  the  batch  sequence  number  and  transferred  to  an  output  buffer  area  within 
the  program.     As  the  program  would  process  each  block  of  data  from  the  tape 
record,   it  would  append  the  edited  data  fields  to  the  data  already  in  this 
buffer.     The  output  data  record,  in  this  "compressed"  format,  was  compatible 
with  the  NAEP  data  entry  system. 

The  program  was  now  prepared  to  cycle  through  the  data  areas 
corresponding  to  the  item  blocks.     The  task  of  translating,  validating,  and 
reporting  errors  for  each  data  field  in  each  block  was  performed  by  a 
subroutine  that  required  only  the  block  identification  code,  the  string  of 
input  data,  and  the  number  of  scorers  who  gridded  the  appropriate 
identification  fields  for  that  block.     This  routine  had  access  to  an  internal 
table  that  had,   for  each  block,  the  number  of  fields  to  be  processed,  and, 
for  each  field,   the  field  type  (alphabetic  or  numeric) ,  the  field  width  in 
the  data  record,  and  the  valid  range  of  values.     The  routine  would  then 
process  each  field  in  sequence  order,  performing  the  necessary  translation, 
validation,  and  reporting  tasks. 

The  first  of  these  tasks  was  checking  for  the  presence  of  hjnphens, 
asterisks  or  question  marks.     Fields  containing  asterisks  and  question  marks 
were  recorded  in  the  audit  file  and  processing  continued  with  the  next  field. 
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No  action  was  taken  on  hyphen-filled  fields  as  that  code  indicated 
nonresporse .     The  field  type  code  dictated  whether  numeric  or  alphabetic 
codes  were  to  be  output  for  a  data  field.     The  next  step  examined  the  type 
code  and  translated  the  input  data  from  alphabetic  to  numeric  if  so 
indicated.     The  field  was  then  validated  for  range  of  response,  recording 
anything  outside  of  that  range  to  the  audit  file.     The  field  type  code  made  a 
further  distinction  among  open-ended  item  scores  and  other  numeric  data 
fields.     If  the  data  field  was  an  open-ended  item  the  routine  used  the  passed 
value  of  the  number  of  scorers  to  determine  if  a  score  should  have  been 
marked.     If  no  scorer  codes  were  indicated  and  the  item  was  marked,  or  a 
scorer  code  was  gridded  and  the  item  was  not  marked,  the  discrepancy  was 
noted  in  the  audit  file.     If  the  current  block  was  either  a  reading  or 
science  block,  the  routine  would  look  ahead  to  the  next  field  for  a  secondary 
scoring  and  compare  its  presence  against  the  absence  of  a  second  scorer  code 
and  vice  versa  and  again  record  a  discrepancy  in  the  audit  file.     Moving  the 
translated  and  edited  data  field  into  the  output  buffer  was  the  last  task 
performed  in  the  processing  loop. 

The  routine  passed  the  edited  data  string  back  to  the  program,  which 
then  appended  it  to  the  current  output  buffer  and  set  up  to  process  the  next 
block  within  the  booklet.     The  completed  string  of  data  was  written  to  the 
data  file,  using  the  batch  sequence  number  as  the  key  for  direct  access  by 
the  entry  system  programs. 

When  the  next  batch  header  record  or  end  of  file  was  encountered  the 
data  and  audit  files  were  closed,  an  audit  listing  was  generated,  and'a  count 
of  the  number  of  records  processed  written  to  the  message  log.     The  tracking 
record  for  that  batch  was  updated  with  the  current  date  and  time  and  the 
record  count,  then  rewritten  to  the  tracking  file. 

When  the  end  of  file  was  reached,  the  program  closed  and  rewound  the 
tape  file,  closed  the  tracking  file,  and  spooled  the  message  log,   the  audit 
listing,  and  the  batch  listing  to  the  printer. 

6.3,3    DATA  ENTRY  AND  RESOLUTION  SYSTEM 

The  resolution  program  was  essentially  the  same  program  used  in  the  1984 
assessment  (see  Rogers,  1987)  for  data  entry,  modified  to  accommodate  changes 
in  the  assessment  design  and  data  entry  operations. 

The  1984  entry  system  was  set  up  to  process  each  age  group  separately, 
as  the  administration  periods  for  each  were  nonoverlapping .     There  were  three 
sets  of  programs,  three  tracking  files,  and  three  areas  for  the  data  and 
control  files.     The  main  part  of  the  1986  assessment,  however,  was 
administered  to  all  three  age  groups  at  the  same  time.     The  new  system  now 
had  to  be  able  to  process  the  materials  from  these  groups  simultaneously:  one 
set  of  programs  using  a  single  tracking  file  to  control  processing  for  three 
data  areas . 

The  program  structure  of  the  system  itself  was  broken  down  into  separate 
programs  for  each  main  function  (school  worksheet  entry,  student  data 
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entry/resolution,  and  questionnaire  entry/resolution),  to  permit  the 
modification  or  enhancement  of  one  component  while  allowing  the  others  to 
operate.    Access  to  these  programs  was  controlled  through  a  menu- type 
procedure  written  in  the  VAX  command  language  and  using  screen  control 
directives . 

The  tracking  files  in  the  1984  entry  system  used  the  session 
identification  code  as  the  index  key  for  direct  access  to  session 
information.     This  proved  to  be  an  unfortunate  choice  for  the  few  times  that 
the  session  code  was  entered  incorrectly  from  the  worksheet  entry  program. 
If  a  session  code  had  to  be  changed,  both  the  tracking  record  and  the  file 
names  of  the  associated  data  and  audit  files  had  to  be  modified,  requiring 
special  programming.     The  adoption  of  a  batch  identification  code  as  the 
index  key  for  the  1986  entry  system  tracking  file  permitted  changing  the 
session  codes  without  impacting  other  parts  of  the  system. 

A  related  problem  frequently  occurred  under  the  1984  entry  system  with 
regard  to  the  student  identification  numbers  in  each  batch  data  file.  These 
codes  were  used  as  the  indox  key  for  direct  access  of  the  student  data 
records  and  an  erroneous  code  was  difficult  to  correct.     In  anticipation  of 
the  many  types  of  problems  involved  in  accurately  gridding  the  student 
identification  numbers  and  transcribing  them  through  the  scanner,  the  batch 
sequence  number  was  adopted  as  the  key  for  the  batch  data  files  in  the  1986 
entry  system.     This  number  was  generated  by  the  scanning  program,  written  to 
each  record  on  the  scanning  output  file,  and,  by  its  nature,  nonredundant . 

Another  addition  to  the  batch  data  records  was  the  data  entry  status 
codes.     The  records  in  a  batch  file  were  generated  in  one  of  two  ways:  the 
tape  loading  program  or  manual  entry  of  the  booklets  rejected  by  the  scanner. 
The  manually  entered  records  had  to  undergo  the  two-step  entry  and 
verification  processes.     Due  to  the  high  accuracy  rate  of  the  optical 
scanner,  the  loaded  records  were  treated  by  the  system  as  if  they  had  been 
through  verification.     The  entry  status  code  was  used  to  distinguish  between 
records  that  were  undergoing  manual  processing  and  those  that  were  loaded. 
The  code  on  each  record  was  tested  and  set  by  the  different  processes:  entry, 
verification,   loading,  and  resolution.     Since  each  record  contained  its  own 
status  code,   it  was  no  longer  necessary  to  maintain  a  vector  of  booklet 
counts  on  the  tracking  record  as  in  the  1984  system. 

The  form  parameters,  which  controlled  processing  of  each  data  entry 
screen,  were  maintained  in  a  text  library.  Each  set  of  parameters  for  each 
form  were  stored  as  a  separate  member  or  subfile  within  this  library.  This 
format  permitted  easy  extraction,  modification  and  replacement  of  parameter 
information  as  well  as  faster  access  by  the  entry  programs.  A  set  of 
programs  were  developed  to  facilitate  the  entry,  documentation,  and  editing 
of  the  form  parameter  data. 

The  teacher  questionnaires  for  the  two  older  grade/ages  contained 
separate  content-related  sections  of  questions  within  and  a  corresponding  set 
of  boxes  on  the  front  cover.     The  teacher  filling  out  a  questionnaire  was 
supposed  to  indicate  the  subject  taught  for  that  grade/age:  reading, 
mathematics,  or  science  at  grade  7/age  13  and  reading,  mathematics,  science, 
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or  history  at  grade  11/age  17.  The 
enhanced  to  accept  the  subject  code 
entry  of  responses  to  the  questions 


teacher  questionnaire  entry  program  was 
and  display  the  appropriate  form  for  the 
for  that  subject. 
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The  data  editing  process  is  divided  into  three  separate  steps: 
validation,  identification,  and  correction.     Validation  ensures  that  each 
data  value  in  the  computer  file  is  of  the  correct  type,  is  within  a  range  or 
set  of  ranges  of  values,  and  is  consistent  with  other  data  values.  All 
invalid  data  values  are  then  identified  and  located  in  the  raw  data.  The 
erroneous  data  are  then  either  corrected  or  flagged  as  unresolvable  in  the 
computer  file. 

The  errors  uncovered  by  the  editing  process  fall  into  two  types:  those 
made  by  the  respondent  (e.g.,  choosing  two  responses  for  a  multiple-choice 
exercise  requiring  only  one  response)  and  those  made  by  data  entry.  The 
validation  process  reports  both  types  of  error  with  no  knowledge  of  their 
source.     The  identification  process  determines  the  type  of  each  error.  The 
data  entry  errors  are,  for  the  most  part,  correctable;  the  correct  value  can 
be  determined  from  an  examination  of  the  raw  data.     Errors  made  by  the 
respondent,  however,  are  difficult,   if  not  impossible,  to  correct.     If  the 
intent  of  the  respondent  cannot  be  determined,  the  error  must  remain 
unresolved,  but  be  flagged  in  some  way  to  prevent  incorrect  interpretation  in 
analysis  and  reporting  procedures. 


6.4.1    ABSENTEE  DATA 

A3  described  in  section  6.1.2,  the  absentee  data  were  transcribed  by  ETS 
key  entry  systems  from  the  administration  schedules.     Key  entry  operation 
standards  required  that  all  data  be  entered  and  verified.  Verification 
consisted  of  a  blind  second  entry  of  each  data  record,  comparing  each  data 
value  with  the  original  entry.     This  procedure  ensured  the  highest  likelihood 
of  correctly  transcribing  the  data. 

The  output  data  files  were  transferred  to  disk  where  the  data  values 
could  be  validated  by  specific  programs  and  corrected  through  the  use  of  an 
online  editor.     Validation  consisted  of  matching  the  school  and  session  codes 
with  those  in  the  tracking  file  and  checking  that  the  sex,  grade,  and  birth 
date  codes  were  within  the  appropriate  ranges  for  age  cohort  and  session 
type.     A  further  check  performed  on  these  files  compared  the  number  of 
absentee  records  within  each  session  against  the  absentee  count  field  on  the 
corresponding  tracking  record. 

The  online  text  editor  proved  to  be  the  easiest  and  most  efficient 
method  of  correcting  most,  if  not  all,  of  the  errors.     The  editor  had  the 
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facility  for  locating  specific  records,  writing  over  the  incorrect  field  with 
its  correct  value,  and  performing  global  substitution  of  systematic  data 
errors,  such  as  incorrect  school  code,  over  one  or  more  records. 

The  corrected  file  was  again  processed  by  the  validation  program  to 
ensure  that  all  errors  had  been  fixed  and  that  no  new  problems  were  created 
in  the  process.     If  more  errors  were  uncovered,  the  cycle  of  identifying  the 
records,  correcting  the  errors,  and  validating  the  corrected  file  was 
repeated  until  no  more  errors  were  found.    At  this  point,   the  absentee  file 
was  ready  for  transmittal  to  Westat  for  sampling  weight  estimation. 


6.4.2    STUDENT  DATA 

The  introduction  of  scannable  materials  to  the  1986  assessment  greatly 
improved  the  efficiency  and  accuracy  of  the  transcription  process  by  removing 
the  possibility  of  human  error.     The  scanning  machinery  was  programmed  to 
detect  the  marked  responses  in  unique  and  fixed  positions  on  each  page; 
erroneous  and  out-of-range  response  codes  could  not  be  generated. 

On  the  other  hand,  removing  human  intervention  as  a  source  of  error  also 
prevented  the  exercise  of  human  judgment  when  more  than  one  mark  was  detected 
for  a  single-response  item.     This  would  commonly  happen  when  a  subject  marked 
a  second  response  without  erasing  the  first,  or  misinterpret  the  question  as 
a  "circle  all  that  apply" -type  response.     Neither  the  human  eye  nor  the 
scanning  'equipment  can  determine  the  student's  intent  in  such  a  situation. 
However,  if  the  student  had  incompletely  erased  the  first  response ,  or 
inadvertently  made  a  stray  mark  on  one  of  the  bubbles,  the  scanning  program 
would  also  return  a  multiple  response  code  where  a  human  eye  could  determine 
the  intended  response.     Hence  there  were  proportionally  many  more  multiple 
response  codes  produced  by  the  scanning  process  in  1986  than  by  the  manual 
entry  process  in  1984. 

Furthermore,  collating  errors  in  the  printing  of  the  booklets  resulted 
in  both  missing  and  multiple  pages,  which  the  scanning  program  was  unprepared 
to  handle.     A  new  code  was  appointed  and  used  to  designate  responses  to  items 
from  pages  that  were  missing  or  otherwise  unscannable. 

Every  multiple  response  code  and  unscannable  page  code  had  to  be  checked 
against  the  raw  data  and,  where  possible,  corrected  by  resolution  operators. 
At  the  completion  of  resolution  processing,  all  of  the  batch  student  data 
files  were  spooled  onto  a  single  master  file  in  preparation  for  transfer  to 
the  IBM  mainframe.     A  second  validation  was  performed  during  this  spooling 
process  to  catch  errors  that  had  "slipped  through"  the  entry  system.  An 
editing  program  was  developed  for  applying  corrections  to  this  master  file, 
using  the  same  methodology  as  for  the  data  entry  program.     This  master  file 
also  served  as  the  basis  for  preliminary  descriptive  data  analyses  and 
quality  control  checks. 
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6.4.3    QUESTIONNAIRE  DATA 


The  data  entry  system  was  used  for  the  entry  of  questionnaire  data  and 
served  as  the  first  line  of  defense  against  bad  data.     As  described  above, 
all  data  values  were  validated  for  type  and  range  as  they  were  entered  from 
the  data  terminal  ke^'^board.     Special  codes  assigned  for  multiple  and 
indeterminate  responses  were  recorded  and  reported  via  the  audit  trail.  The 
indeterminate  values  were  later  corrected  under  the  resolution  process! 

The  questionnaire  files  received  the  same  secondary  validation 
processing  as  the  student  data.     Special  attention  was  given  to  the  "circle 
all  that  apply"  items  to  ensure  consistency  in  the  coding  of  responses:     if  a 
respondent  circled  one  or  more  of  the  alternatives,  those  would  be  coded  "1" 
while  the  rest  would  be  coded  "0";   if  no  alternatives  were  marked,  yet  the 
respondent  had  the  opportunity  to  reply,  all  fields  would  be  coded  "0";   if  no 
alternatives  were  marked  and  the  respondent  had  not  reached  the  item  or  was 
instructed  to  skip  it,  all  fields  would  be  coded  as  "no  response". 


6.4.4    PROFESSIONALLY  SCORED  ITEMS 

The  open-ended  reading,  mathematics,  and  science  item  responses  were 
read  and  scored  prior  to  scanning  processing.     Their  data  values  were  subject 
to  the  same  editing  procedures  as  the  multiple -choice  item  responses.  The 
open-ended  computer  items,  however,  were  not  scored  until  after  scanning  and 
resolution  processing.     It  was  neither  feasible  nor  economically  prudent  to 
enter  so  few  scores  for  each  booklet  through  the  entry  system,   therefore  this 
data  went  through  a  separate  entry  and  editing  process. 

Special  forms  were  designed  and  printed  for  each  age  group  for  the 
scoring  of  the  open-ended  computer  items.     As  each  booklet  was  scored,  the 
student  ID  number,  birth  date,  and  PSU  and  school  codes  were  transferred  to 
the  form.     The  first  set  of  scores  were  written  into  the  appropriate  boxes 
for  the  items  in  that  booklet.     Every  fifth  line  on  the  form  had  boxes  for 
the  entry  of  the  second  reader's  scores,  ensuring  a  20  percent  rater 
reliability  sample. 

These  forms  were  batched  and  forwarded  to  ETS  key  entry  systems  where 
they  were  entered,  verified,  and  transcribed  to  magnetic  tape.     These  tape 
files  were  loaded  onto  the  VAX  computer  system  where  specially  written 
validation  programs  performed  thorough  checks  on  the  data  values.     An  online 
text  editor  was  used  to  make  corrections  to  the  data  and  the  validation 
programs  run  again.     When  all  of  the  items  had  been  scored,  entered, 
validated,  and  corrected,  the  data  files  were  transferred  to  the  IBM 
mainframe  system  for  merging  with  the  student  database. 


6.4.5  CONCLUSION 


Before  the  NAEP  data  entry  methodology  was  developed,  the  editing 
process  for  any  data  file  proceeded  in  the  same  manner  as  for  the  absentee 
data  and  professionally  scored  computer  items.     The  validation  process  was 
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especially  inefficient  because  it  was  performed  after  the  fact  of 
transcription  and  often  by  a  second  party  who  did  not  have  immediate  access 
v"o  the  raw  data.     Putting  the  validation  mechanism  at  the  point  of  entry 
removed  most,  if  not  all,  of  this  inefficiency  by  informing  the  entry 
operator  of  a  possible  keying  error  while  the  raw  data  value  was  accessible. 

The  editing  process  does  not  guarantee  that  all  errors  are  removed  from 
the  data;  only  that  the  invalid,  inconsistent,  or  otherwise  unreasonable 
values  have  been  at  least  identified,  if  not  corrected.     If  a  data  value  has 
been  miskeyed  during  the  entry  process  and  meets  the  validation  criteria, 
this  error  could  persist  through  the  editing  process  to  the  analysis  stage 
without  detectioii.     The  verification  process  detects  most  of  these  errors  by 
comparing  independent  entries  of  the  same  data  and  reporting  discrepancies. 
The  likelihood  of  an  error  surviving  verification  is  thus  very  small,  but 
still  present.    A  quality  control  process  must  follow  the  entry  and  editing 
processes  to  ensure  that  the  data  values  in  a  given  record  agree  with  the 
responses  in  the  corresponding  instrument. 
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Chapter  6.5 
QUALITY  CONTROL  OF  DATA  ENTRY 


John  J.  Ferris 
Educational  Testing  Service 


The  purpose  of  this  work  was  to  assess  the  accuracy  of  the  NAEP  1986 
data  entry  operation,  or  how  closely  the  contents  of  the  various  instruments 
matched  the  corresponding  data  in  the  resulting  datasets  that  were  analyzed 
A  more  complete  discussion  of  the  general  approach  taken  to  quality  control 
can  be  found  in  Chapter  10.5  of  the  Technical  Report  for  the  1984  assessment 
Following  are  the  detailed  results  of  the  quality  control  operation  for  each 
of  the  1986  instruments;  a  table  at  the  end  of  this  chapter  summarizes  the 
findings . 


6.5.1    STUDENT  DATA 

A  total  of  213  different  booklets  was  used  for  the  student  assessment 
across  the  three  cohort  levels.     These  booklets  were  designed  to  be  machine- 
scanned,  but  not  all  booklets  were  received  in  scannable  condition.  Those 
that  could  not  be  scanned  had  to  be  keyed  by  hand. 

One  of  each  of  the  213  different  booklets  from  among  those  that  were 
scanned  was  selected  for  analysis.     In  addition  it  was  considered  prudent  to 
treat  the  hand-keyed  booklets  as  an  independent  pool;  therefore,  one  each  of 
them  was  also  selected  for  quality  control  analysis.     The  booklets  that 
belonged  to  the  Language  Minority  Probe  in  this  assessment  were  not 
considered  independently  since  the  booklets  used  were  the  same  and  the  data 
entry  process  was  the  same. 

At  Age    9,  52  booklets  were  checked  out  of  37,401  scanned; 

52        "  "  "  «       ..     ]^  068  keyed. 

At  Age  13,  68        ••  «  ><       "  43,900  scanned; 


68        "  " 


886  keyed. 


At  Age  17,  93   -  48,164  scanned; 


93 


"     L,162  keyed. 


TOTALS:  213  scanned  booklets  were  checked  (0.16  percent) 

213  keyed  "  •■  "        (6.8  percent) 
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As  expected,  the  scanned  booklets  were  much  more  accurately  captured  by  the 
data  entry  process;  only  one  "error"  was  discovered  in  all  213  of  these 
booklets,  the  pickup  of  an  erasure.     In  the  group  of  keyed  booklets  examined, 
a  total  of  11  keystrokes  were  wrong.     The  total  number  of  keystrokes  in  a 
full  set  of  213  booklets  was  42,366. 

6.5.2  EXCLUDED  STUDENT  QUESTIONNAIRE  DATA 

Excluded  student  questionnaires  were  randomly  sampled  at  the  rate  of  2.5 
percent,  the  same  rate  used  in  the  1984  assessment: 

At  Age    9,     53  booklets  checked  out  of  2151 

"     "    13,     64      "  "     "  2595 

"     "     17,  _61      "  "  "     "  2465 

TOTALS:        178      "  «  «     ..  7211     (2.5  percent) 

No  errors  at  all  were  found  among  these  178  booklets ,  which  involved  a  total 
of  16,020  keystrokes. 

6.5.3  TEACHER  QUESTIONNAIRE  DATA 

Teacher  questionnaires  were  randomly  sampled  at  the  rate  of  2.5  percent, 
or  approximately  one  booklet  out  of  40. 

At  Age    9,  19  booklets  checked  out  of  789 

..     .,    13,  19      "  "  "     "  800 

n     ..     X7,  29      "  "  "     "  1244 

TOTALS:  67      "  "  "     "  2833     (2.4  percent) 

A  total  of  25,809  keystrokes  was  involved  in  this  sample  of  teacher 
questionnaires.     One  of  these  keystrokes  was  in  error. 

A  separate  teacher  questionnaire  was  administered  as  part  of  the 
Language  Minority  Probe  in  the  1986  assessment.     These  booklets  were  sampled 
at  about  the  same  rate. 
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At  Age    9,  4  booklets  checked  out  of  168 

13,  3        "  "  "     "  135 

17,  _2        "  "  "  92 

TOTALS:  9        "  "  "     «  395     (2,3  percent) 

A  total  of  1,589  keystrokes  was  involved  in  this  sample  of  teacher 
questionnaires.     One  of  these  keystrokes  was  in  error. 

6,5.4     SCHOOL  CHARACTERISTICS  QUESTIONNAIRE  DATA 

School  characteristics  questionnaires  were  randomly  sampled  at  the  rate 
of  5  percent,  or  one  booklet  out  of  20. 

At  Age    9,  29  booklets  checked  out  of  598 

"     "     13,  26        "  "  «     «  531 

"     "     17,  20        "  "  "     "  402 

TOTALS:  75  «  "     "  1531     (4,9  percent) 

A  total  of  18,739  keystrokes  was  involved  in  this  sample  of  school 
questionnaires.     Two  of  these  keystrokes  were  in  error. 

A  separate  school  questionnaire  was  administered  as  part  of  the  Language 

Minority  Probe  in  the  1986  assessment.  These  booklets  were  sampled  at  about 
the  same  rate. 

At  Age     9,       5  booklets  checked  out  of  106 

"     "     13,      3        "  "          "     "  50 

"     "     17,    _3        "  »          "     "  26 

TOTALS:          11        "  "          «     »  182     (6.0  percent) 

A  total  of  7,029  keystrokes  was  involved  in  this  sample  of  school 
questionnaires.     No  errors  were  found. 
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6.5.5    COMPUTER  COORDINATOR  QUESTIONNAIRE  DATA 


Computer  coordinator  questionnaires  were  randomly  sampled  at  the  rate  of 
5  percent,  or  one  booklet  out  of  20. 

At  Age    9,     13  booklets  checked  out  of  250 

N       n  14  "  .inn  287 

«  17,     17        "  "  "     "  336 

TOTALS:  44        "  «  «     «        873    (5.0  percent) 

A  total  of  8,316  keystrokes  was  involved  in  this  sample  of  computer 
coordinator  questionnaires.     Two  of  these  keystrokes  were  in  error, 

6.5.6    SUMMARY  OF  RESULTS 

The  quality  control  of  the  NAEP  data  for  1986  showed  extremely  high 
standards  of  data  entry.     The  use  of  scannable  booklets  contributed  to  the 
improvement  of  what  had  already  been  a  very  high  quality  of  data  entry  in  the 
1984  assessment  (see  Ferris,  1987).     Two  values  are  tabled  below,  the 
observed  error  rate  and  the  upper  bound  of  the  99.8  percent  confidence 
interval . 


ERLC 


-146- 

l4o 


Table  6.5.1 

Observed  Error  Rate  and  Upper  Confidence  Limit,  by  Instrument 


OBSERVED 

INSTRUMENT  ERROR  RATE 

Student  Data  including 

Language  Minority  -  scanned 

(97.7  percent  of  books)  .00002 

Student  Data  including 

Language  Minority  -  keyed 

(2.3  percent  of  books)  .00026 

Excluded  Student  Questionnaire  zero 

Teacher  Questionnaire  .00004 

Language  Minority  Teacher 

Questionnaire  ^  00065 

School  Questionnaire  .00011 

Language  Minority  School 

Questionnaire  zero 

Computer  Coordinator  Questionnaire  .00024 


UPPER  99.8% 
CONFIDENCE  LIMIT 


,0002 


.0005 
.0004 
.0003 

.0055 
.0006 

.0015 
.0013 


Note:  An  error  was  discovered  in  the  computation  of  the  corresponding 
values  for  the  student  data  in  the  1984  assessment.  The  correct  values  for 
that  assessment  are: 


Student  Data 
(not  as  reported: 


.00006 
.00002 


.0003 
.0001) 
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Chapter  6.6 
DATABASE  CREATION 


Alfred  M.  Rogers 
Educational  Testing  Service 


The  data  transcription  and  editing  procedures  described  in  Chapter  6  1 
resulted  in  the  generation  of  disk  and  tape  files  containing  various 
assessment  information.     Before  any  analysis  could  begin,  these  files  had  to 
be  pulled  together  into  a  comprehensive,  integrated  database.  Sampling 
weights  were  also  required  in  order  to  make  any  valid  statistical  inferences 
about  the  population  from  which  the  assessment  sample  was  drawn. 

This  chapter  describes  the  processes  of  extraction  of  sample  information 
for  the  derivation  of  sampling  weights,  and  the  merging,  or  bringing 
together,  of  the  many  transcription  files  into  the  NAEP  database. 


6.6.1  EXTRACTION 

_        For  each  grade/age  cohort,  four  sets  of  weights  were  required  to  perform 
inferential  analyses:  school  weights,  excluded  student  weights,  student 
weights,   and  teacher  weights.     Due  to  the  method  by  which  teachers  were 
selected,  sampling  weights  could  not  be  assigned  to  teachers,  but  were 
instead  assigned  to,  sfiidents  who  were  linked  to  participating  teachers  (See 
Chapter  3  for  more  details.) 

All  of  the  sample  information  was  extracted  from  the  data  files  edited 
and  transferred  to  tape  files  for  shipment  to  Westat,  where  the  weight 
computation  was  performed.     The  editing  process  included  both  the  validation 
of  the  data  values  as  well  as  frequency  distribution  analyses  to  be  compared 
with  tracking  information  from  the  data  entry  system. 

The  school  sample  information  was  available  to  Westat  from  the  beginning 
of  the  assessment.     They  did  not  require  any  additional  information  from  ETS 
to  compute  school  sample  weights. 

The  excluded  student  sample  information  was  extracted  from  the  excluded 
student  questionnaire  data  file.     This  information  included  booklet  serial 
number,  PSU  and  school  code,  grade,  sex,  birth  date,  race/ethnicity,  and  a 
code  indicating  reason  for  exclusion.     All  data  fields  were  taken  from  the 
front  cover  information  of  each  booklet,  except  for  the  exclusion  code  which 
was  derived  from  the  response  to  item  3  of  the  questionnaire.     A  listing  of 
the  excluded  student  questionnaires  that  had  not  been  received  at  ETS  was 
included  with  the  file  for  each  grade/age  cohort. 
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The  student  sample  information  came  from  two  sources:     the  student 
database  and  the  absentee  file  from  the  administration  schedules.  The 
assessed  student  sample  information  included  booklet  serial  number,  PSU  and 
school  code,  grade,  sex,  birth  date,  race/ethnicity,  and  teacher  code.  Since 
the  absent  students  were  not  observed  and  not  assigned  an  assessment  booklet, 
the  booklet  serial  number,  race/ethnicity,  and  teacher  code  were  not 
available  for  the  absentee  data. 

The  absentee  file  had  to  be  adjusted  for  makeup  sessions.     The  field 
administration  procedures  required  scheduling  of  makeup  sessions  if  absentee 
rates  exceeded  certain  limits.     The  students  attending  these  makeup  sessions 
were  supposed  to  be  originally  sampled  students  who  were  absent  for  the 
regular  sessions.     Failure  to  remove  the  makeup  students  from  the  absentee 
file  would  have  resulted  in  incorrect  estimates  of  the  number  of  students  in 
those  schools.     This  problem  could  have  been  particularly  acute  in  the  grade 
11/age  17  sample  where  absentee  rates  were  high  and  many  schools  required 
makeup  sessions. 

The  first  step  in  the  removal  process  was  to  identify  the  students  in 
the  student  file  who  attended  makeup  sessions  in  each  school.     Then,   for  each 
school  and  session  type  (spiral  or  tape),  the  sex,  grade,  and  birth  dates  of 
the  makeup  students  were  matched  with  those  of  the  absentee  students  in  the 
same  school  and  session  type.     The  absentees  identified  by  perfect  matches 
were  removed  from  the  absentee  file;  the  remaining  unmatched  makeup  students, 
if  any,  were  paired  with  randomly  selected  absentees  who  were  then  removed 
from  the  file.     This  latter  procedure  was  necessary  only  for  the  grade  11/age 
17  sample  in  only  a  few  of  the  many  schools  that  had  makeup  sessions. 

The  teacher  sample  information  was  extracted  from  the  teacher 
questionnaire  data  file.     It  consisted  of  only  the  PSU,  school,  and  teacher 
codes  from  the  questionnaire  booklet  covers.     Westat  used  this  information  in 
conjunction  with  the  student  sample  information  to  produce  a  file  of  student- 
based  teacher  weights. 

6.6.2    FILE  MERGING 

The  transcription  process  resulted  in  the  generation  of  five  data  files 
for  each  grade/age  cohort:     one  file  for  each  of  the  three  questionnaire 
instruments,  the  student  response  data  file  from  the  data  entry  system,  and 
the  student  reading  and  writing  scores  from  professional  scoring  and  key 
entry.     The  sample  weight  derivation  process  produced  an  additional  four 
files  of  sampling  weights.     To  perform  data  analysis,  these  files  had  to  be 
integrated  into  a  coherent  and  comprehensive  database. 

This  database  would  ultimately  consist  of  four  files  per  cohort: 
school,  teacher,   excluded  student,  and  student  files.     The  student  file  would 
contain  all  six  student  samples:     the  spiral  and  five  bridge  samples.  The 
school  file  could  be  linked  to  the  other  three  files  through  the  PSU  and 
school  codes.     The  teacher  file  could  be  linked  to  the  student  spiral  sample 
through  the  PSU,  school  and  teacher  codes. 
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^      The  school  file  vas  created  by  merging  the  school  questionnaire  file 
^^l^^^o^P^^®^  coordinator  questionnaire  and  then  with  the  school  weights 
file.    The  PSU  and  school  code  were  used  as  the  matching  criterion  Each 
record  of  the  resulting  file  was  formed  by  concatenating  the  weight 
information  with  the  response  data.     Since  not  all  schools  returned  their 
questionnaires,  some  of  the  output  records  contained  only  weight  information. 

The  teacher  file  was  generated  from  the  teacher  questionnaire  file 
Since  the  teacher  weights  were  derived  at  the  student  level,  no  information 
had  to  be  added  to  tha  questionnaire  data. 

The  excluded  student  file  was  the  result  of  merging  the  excluded  student 
questionnaire  file  with  the  excluded  student  weights  file.     The  booklet 
serial  number  was  used  as  the  matching  criterion. 

The  creation  of  the  student  data  file  was  a  three-stage  process  merging 
the  student  weights,   teacher-based  student  weights, and  professionally  scored 
computer  items  with  the  student  response  data,   in  tha-  order.     In  all  three 
procedures,   the  booklet  serial  number  was  used  as  the  matching  criterion 
The  merging  of  the  professionally  scored  item  data  was  a  more  complex 
procedure  than  the  others,  because  the  set  of  scores  for  each  item  within  a 
booklet  were  inserted  into  the  response  data  fields  in  the  order  in  which  the 
Items  appeared  in  the  booklet. 

The  database  was  then  ready  for  analysis.     As  new  data  values  and  scores 
were  derived,  they  were  added  to  the  relevant  files  using  the  same  matching 
procedures  as  described  above.     The  public -use  data  tapes  files  were 
ultimately  generated  from  this  database. 


6.6.3    MASTER  CATALOG 

A  critical  part  of  any  database  is  the  processing  control  and 
descriptive  information.     A  central  repository  of  this  information  may  be 
accessed  by  all  analysis  and  reporting  programs  to  provide  correct  parameters 
for  processing  the  data  fields  as  well  as  consistent  identification  labeling 
of  the  analysis  results.     The  master  catalog  file  was  designed  and 
constructed  to  serve  both  of  these  purposes. 

Each  record  cf  'he  master  catalog  contains  the  processing,  labeling 
classification,  an.1  location  information  for  each  data  field  in  the  database 
The  control  parameters  are  used  by  the  access  routines  in  the  analysis 
programs  to  define  the  manner  in  which  the  raw  data  values  are  to  be 
transformed  and  processed. 

All  data  fields  have  a  50 -character  label  in  the  catalog  describing  the 
contents  of  the  field  and,  where  applicable,   the  source  of  the  field  The 
data  fields  with  discrete  or  categorical  values  have  additional  label  fields 
in  the  catalog  containing  the  permitted  values  and  8-  and  20-  character 
labels  for  those  values . 
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The  classification  area  of  the  catalog  record  contains  distinct  fields 
corresponding  to  predefined  classification  categories  for  the  data  fields. 
For  a  given  classification  field,  a  nonblank  value  indicates  the  code  within 
that  classification  category  for  the  data  field.     This  permits  the  collection 
of  identically  classified  items  or  data  fields  by  performing  a  selection 
process  on  one  or  more  classification  fields  in  the  catalog. 

According  to  the  NAEP  design,  it  is  possible  for  item  data  fields  to 
occur  in  more  than  one  age  assessment  and  more  than  one  block  within  each 
age.     The  location  fields  of  the  catalog  record  contain  the  age,  block  and, 
where  applicable,  the  item  sequence  number  within  block  of  each  occurrence  of 
the  data  field  throughout  the  1986  database. 

The  master  catalog  file  was  constructed  in  parallel  with  the  collection 
and  transcription  of  the  assessment  data  to  be  ready  for  use  by  analysis 
programs  when  the  database  was  created.    As  new  data  fields  were  derived  and 
added  to  the  database,  their  descriptive  and  control  information  was  entered 
into  the  catalog. 

One  of  the  most  important  uses  of  the  master  catalog  was  the  control  of 
the  creation  of  the  public -use  data  tapes  files  as  well  as  the  codebooks  and 
file  layouts.     A  synopsis  of  this  process  is  presented  in  the  next  chapter. 
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Chapter  6.7 
PUBLIC-USE  DATA  TAPE  CONSTRUCTION 


Alfred  M.  Rogers 
Educational  Testing  Service 


The  public-use  data  tapes  are  designed  to  permit  any  researcher  or 
research  organization  with  an  interest  in  the  National  Assessment  to  perform 
secondary  analysis  on  the  same  data  as  that  used  at  ETS.     This  section 
discusses  some  of  the  issues  raised  during  the  creation  of  the  data,  and 
stimmarizes  the  procedures  followed  in  generating  the  data  and  related 
materials . 

The  three  elements  of  the  distribution  package  are  the  data  tapes, 
printed  documentation,  and  microfiche  of  the  assessment  instruments.  Each 
grade/age  cohort  is  represented  on  a  separate  tape,  with  each  tape  containing 
the  data  files;  a  set  of  SPSS-X  control  statement  files  for  generating  an 
SPSS-X  system  file  for  each  data  file;  a  set  of  SAS  control  statement  files 
for  generating  a  SAS  system  file  for  each  data  file;  and  a  set  of  machine- 
readable  catalog  files  containing  control  and  descriptive  information  for 
each  data  file,  for  the  non- SPSS-X  and  non-SAS  user.     The  printed 
documentation  consists  of  four  volumes:  a  guide  to  the  use  of  the  data  files, 
and  a  set  of  file  layouts  and  codebooks  for  the  data  files  within  each  of  the 
three  cohorts  (see  The  NAEF  1985-86  Public-Use  Data  Tapes  Version  2.0  Users^ 
Guide  [Rogers,  Kline,  Norris,  Johnson,  Mislevy,  ^vick,  Barone,  6c  Kaplan 
1988]).  »  F  » 


6.7.1    FILE  DEFINITION 

The  organization  and  format  of  the  data  files  to  be  produced  was  the 
first  issue  to  be  addressed.     The  ETS  database  consisted  of  four  data  files 
for  each  grade/age  cohort,  corresponding  to  the  three  questionnaire 
instruments  and  the  student  database,  incorporating  the  spiral  and  all  five 
bridge  samples.     The  logical  relationship  of  the  data  files  was  a  three-level 
hierarchy,  with  the  six  student  and  the  excluded  student  samples  at  the 
bottom  level;  the  teacher  sample  at  the  next  level,  with  a  linkage  only  to 
the  spiral  sample;  and  the  school  sample  at  the  top,  with  direct  linkages  to 
all  samples  below  it.     A  linkage  may  be  viewed  as  a  one -to -many  mapping  of 
the  records  within  the  two  files  linked.     For  example,  one  school  record  is 
linked  to  one  or  more  records  in  the  teacher  file,  and  each  of  these  teacher 
records  are  in  turn  linked  to  one  or  more  records  in  the  spiral  student  file. 

One  organization  scheme  has  seven  files  corresponding  to  the  seven 
samples  at  the  bottom  level,  with  the  data  from  the  higher-order  samples 
appended  to  and  repeated  across  as  many  of  thf  lower- level  records  as 
required  by  the  linkages.     Using  the  previous  example,  each  spiral  sample 
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record  would  be  appended  by  its  corresponding  teacher  record  and  school 
record.     This  approach  places  no  demand  on  the  user  to  define  the  linkages 
since  each  data  record  is  complete,  but  it  requires  substantially  more 
computer  storage  space  due  to  the  larger  record  size. 

An  alternative  scheme  would  have  these  same  seven  samples  without  the 
appended  teacher  and  school  data.     The  teacher  and  school  samples  would 
reside  in  their  own  files,  with  special  data  fields  in  all  files  to 
facilitate  their  linkage  through  program  control.     At  the  expense  of  a  little 
more  sophistication  on  the  part  of  the  user,  this  approach  is  more  economical 
in  computer  resource  utilization.     This  potential  for  savings  on  computer 
storage  and  processing  costs  was  the  overriding  consideration  in  choosing 
this  scheme . 


6.7.2    VARIABLE  DEFINITION 

The  selection  and  arrangement  of  variables,  or  data  fields,   in  each  file 
was  the  next  order  of  business.     The  first  step  in  the  decision  process  was 
the  generation  of  a  file  of  variable  descriptors  for  each  data  file  to  be 
created.     Each  of  these  LABELS  files  contained  one  record  for  each  variable, 
each  record  containing  the  variable  name,  a  short  text  description  of  the 
variable,  and  processing  control  information  to  be  used  by  later  steps  in  the 
public-use  data  tapes  process.     This  file  could  be  edited  for  deletion  of 
variables,  modification  of  control  parameters,  or  reordering  of  the  variables 
within  the  file. 

The  first  program  in  the  processing  stream,  GENLYT,  produced  a  printed 
layout  for  each  file  from  the  information  in  its  corresponding  LABELS  file. 
These  layouts  were  initially  reviewed  for  the  selection  and  ordering  of  the 
variables.     The  variables  that  were  excluded  from  public-use  data  tape 
processing  fell  primarily  into  two  categories:     nonapplicable  and 
confidential . 

The  nonapplicable  variables  were  found  mostly  in  the  student  database. 
Since  the  bridge  samples  were  combined  with  the  spiral  sample,  many  of  the 
variables  that  applied  to  the  spiral  students  did  not  apply  to  the  bridge 
students,  and  vice  versa.     For  example,  the  teacher  code  and  the  student - 
based  teacher  weights  were  used  for  the  analysis  of  spiral  sample  data,  but 
were  not  in  the  design  at  all  for  the  bridge  sample. 

The  confidential  variables  included  any  descriptor  or  code  that  could  be 
used  to  identify  individual  states,  schools,  or  students  in  the  NAEP  sample. 
The  PSU,  school,   teacher,  and  student  identification  codes  used  internally  by 
ETS  and  WESTAT  were  "scrambled"  according  to  specific  algorithms  to  obtain 
new  codes  for  use  in  linking  the  files  together. 

Another  confidentiality  problem  arose  in  the  response  data,  where  the 
students  were  asked  to  identify  the  state  they  had  lived  in  four  years  ago. 
A  new  variable  was  created  using  the  response  code  and  current  state 
residency  information  from  the  PSU  code  to  indicate  if  the  student  had  lived 
in  the  same  state,  the  same  region,  or  a  different  region. 
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The  ordering  of  the  variables  within  the  data  files  followed  a  general 
trend  of  decreasing  likelihood  of  usage:     Identification  information  preceded 
weights,  scores,  and  other  derived  variables,  which  were  followed  by  the 
response  data.     The  identification  variables  were  generally  those  on  the 
front  covers  of  the  instruments.     The  derived  variables  included  the  sampling 
weights,   IRT  scale  values,  and  variables  derived  from  the  response  data  or 
other  sources  for  reporting  purposes.     The  response  data  variables  were 
arranged  according  to  their  order  in  the  instrument. 

The  spiral  sample  posed  an  additional  problem  because  it  entailed  the 
expression  of  as  many  as  91  different  booklet  formats  into  a  single  fixed 
format.     The  solution  lay  in  arranging  the  data  "blocks in  order  within 
subject  areas.     The  common  background  questionnaire  preceded  the  first  s^Diral 
block  m  the  new  record.     Each  data  record  from  the  input  student  base  was 
reformatted  according  to  its  booklet  number;   the  data  for  its  constituent 
blocKS  were  moved  into  their  assigned  locations  in  the  output  record  The 
remaining  data  block  areas  contained  blank  fields,   indicating  that  the  data 
was  missing  by  design. 

The  spiral  design  also  created  a  problem  from  the  user's  standpoint- 
how  to  determine,   from  a  given  booklet  record,  which  data  blocks  were  present 
and  their  relative  order  in  the  instrument.     This  problem  was  remedied  by  the 
creation  of  a  set  of  control  variables,  one  for  each  block,  which  indicated 
not  only  the  presence  or  absence  of  the  block  but  its  order  in  the 
instrument.     These  control  variables  were  included  in  the  section  of  derived 
variables . 


6.7.3     DATA  DEFINITION 

To  enable  the  data  files  to  be  processed  on  any  computer  system  using 
any  procedural  or  programming  language,   it  was  desirable  that  the  data  be 
expressed  in  numeric  format.     This  was  possible,  but  not  without  the  adoption 
of  certain  conventions  for  re-expressing  the  data  values. 

As  mentioned  in  Chapter  6.3,   the  responses  to  all  multiple- choice  items 
were  transcribed  and  stored  in  the  database  using  the  letter  codes  printed  in 
the  instruments.     This  scheme  afforded  the  advantage  of  saving  storage  space 
for  Items  wiM-  ten  or  more  response  options,  but  at  the  expense  of 
translating  these  codes  into  their  numeric  equivalents  for  analysis  purposes 
The  response  data  fields  for  most  of  these  items  would  require  a  simple 
alphabetic- to -numeric  conversion.     However,   the  data  fields  items  with  ten  or 
more  response  choices  would  require  "expansion"  before  the  conversion  since 
the  numeric  value  would  require  two  column  positions.     One  of  the  processing 
control  parameters  on  the  LABELS  file  indicates  whether  or  not  the  data  field 
IS  to  be  expanded  before  conversion  and  output. 

The  ETS  database  contained  special  codes  to  indicate  certain  response 
conditions:     no  response,   "I  don't  know"  response,  multiple  response  and 
unresolvable  response.     The  primary  trait  scores  for  the  reading  essay  items 
included  additional  special  codes  for  ratings  of  "illegible"  and  "off-task" 
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by  the  scorers.     A  final  special  code  was  assigned  to  the  items  that,  due  to 
printing  error,  did  not  appear  in  some  of  the  booklets  at  all.    These  codes 
had  to  be  re -expressed  in  numeric  format. 

A  convention  used  by  ECS  in  the  creation  of  their  public-use  data  tapes 
was  adopted  and  enhanced  in  the  designation  of  these  codes:     The  "I  don't 
know"  and  nonrateable  response  was  always  coded  as  7;  the  "no  response"  code 
was  8;  and  the  multiple  response  received  a  code  of  9.    The  small  number  of 
out-of -range  and  "missing"  responses  were  coded  as  blank  fields, 
corresponding  to  the  "missing  by  design"  designation. 

This  coding  scheme  created  conflicts  for  those  items  that  had  seven  or 
more  valid  responses  as  well  as  the  "I  don't  know"  response.     Tnese  items 
also  required  expansion  to  accommodate  the  valid  responses  values.  The 
special  codes  were  "extended"  to  ffll  the  output  data  field:     The  "I  don't 
know"  and  nonrateable  code  was  extended  from  7  to  77;  the  "no  response"  code 
from  8  to  88;  the  multiple  response  code  from  9  to  99. 

The  niimeric  variables  on  the  tape  files  were  classified  into  two 
categories:     continuous  and  discrete.     The  continuous  variables  include  the 
weights,  IRT  values,  identification  codes,  and  item  responses  where  counts  or 
percentages  were  requested.    The  discrete  variables  include  those  items  for 
which  each  numeric  value  corresponds  to  a  response  category.  This 
designation  also  includes  those  derived  variables  to  which  numeric 
classification  categories  have  been  assigned.     The  open-ended  items  were 
treated  as  a  special  subset  of  the  discrete  variables  and  were  given  a 
separate  categorization  to  facilitate  their  identification  in  the 
documentation. 


6.7.4    DATA  FILE  LAYOUTS 

The  data  file  layouts,  as  mentioned  above,  were  the  first  user  product 
to  be  generated  in  the  public-use  data  tapes  process.     The  generation 
program,  GENLYT,  used  a  LABELS  file  as  input  and  produced  a  printable  file. 
This  LAYOUT  file  is  little  more  than  a  formatted  listing  of  the  LABELS  file. 

Each  line  of  the  LAYOUT  file  contains  the  following  information  for  a 
single  data  field:     sequence  number,  field  name,  output  column  position, 
field  width,  niimber  of  decimal  places,  data  type,  value  range,  key  or  correct 
response  value,  and  a  short  description  of  the  field.     The  sequence  number  of 
each  field  is  implied  from  its  order  on  the  LABELS  file.     The  field  name  is 
an  8 -character  label  for  the  field  that  is  to  be  used  consistently  by  all 
public -use  data  tapes  materials  to  refer  to  that  field  on  that  file.  The 
output  column  position  is  the  relative  location  of  the  beginning  of  that 
field  on  each  record  for  that  file,  using  bytes  or  characters  as  the  unit  of 
measure.     The  field  width  indicates  the  munber  of  columns  used  in 
representing  the  data  values  for  a  field.     If  trie  field  contains  continuous 
numeric  data,  the  number  of  decimal  places  value  indicates  how  many  places  to 
shift  the  decimal  point  before  processing  data  values. 
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The  data  type  category  uses  three  codes  to  designate  the  nature  of  the 
data  in  the  field:     Continuous  numeric  data  are  coded  "C";  discrete  numeric 
data  are  coded  "D";  open-ended  item  data  are  coded  "0".    Additionally,  the 
discrete  numeric  fields  that  include  "I  don't  know"  response  codes  are  coded 
-DI"  and  the  open-ended  items  that  include  nonrateable  response  codes  are 
coded  "01".     If  the  field  type  is  discrete  numeric,  the  value  range  is  listed 
as  the  minimum  and  maximum  permitted  values  separated  by  a  hyphen  to  indicate 
range.     If  the  field  is  a  scorable  item  response,  the  correct  response  value, 
or  key,   is  printed.     A  range  of  correct  responses  was  indicated  for  those 
professionally  scored  items  that  received  cut-point  scoring  for  IRT  scaling. 
Finally,  each  variable  was  further  identified  by  a  50-character  descriptor.' 


6.7.5    DATA  FILE  CATALOGS 

The  LABELS  file  contains  sufficient  descriptive  information  for 
generating  a  brief  layout  of  the  data  file.    However,  to  generate  a  complete 
codebook  document,  substantially  more  information  about  the  data  is  required. 
'::his  function  is  filled,  in  part,  by  the  CATALOG  file. 

The  CATALOG  file  is  created  by  the  CATGEN  program  from  the  LABELS  file 
and  the  1986  master  catalog  file.     Each  record  on  the  LABELS  file  generates  a 
CATALOG  record  by  first  retrieving  the  master  catalog  record  corresponding  to 
the  field  name.     The  master  catalog  record  contains  usage,  classification, 
and  response  code  information.     This  record  is  prefixed  by  the  positional' 
information  from  the  LABELS  file:     field  sequence  number,  output  column 
position,  and  field  width. 

The  response  code  information,  also  referred  to  as  "foils",  consists  of 
the  possible  data  values  for  the  discrete  numeric  fields,  and  a  20-character 
description  of  each.     The  CATGEN  program  uses  additional  control  information 
from  the  LABELS  file  to  determine  if  extra  foils  should  be  generated  and 
saved  with  each  CATALOG  record.     The  first  flag  controls  generation  of  the  "I 
don't  know"  or  nonrateable  foil;  the  second  flag  regulates  "no  response"  foil 
generation;  and  the  third  flag  denotes  the  possibility  of  multiple  responses 
for  that  field  and  sets  up  an  appropriate  foil.     All  of  these  control 
parameters,  including  the  expansion  flag,  may  be  altered  in  the  LABELS  file 
by  use  of  a  text  editor  to  suit  the  data  behavior  for  any  given  field. 

The  LABELS  file  supplies  control  information  for  many  of  the  subsequent 
public-use  data  tapes  processing  steps.     The  CATALOG  file  provides  the  detail 
information  for  those  same  steps  and  for  others  as  well. 


6.7.6  CODEBOOKS 

The  data  file  codebook  is  designed  as  a  printed  document  containing 
complete  descriptive  information  for  each  data  field.     Most  of  this 
information  derives  from  the  CATALOG  file;  the  remaining  data  came  from  two 
other  files:     the  COUNTS  file  and  the  IRT  parameters  file. 
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Each  data  field  receives  at  least  one  line  of  descriptive  information  in 
the  codebook.     If  the  data  type  is  continuous  numeric,  no  more  detail  is 
given.     If  the  variable  is  discrete  numeric,  the  codebook  lists  the  foil 
codes,  foil  labels,  and  frequencies  of  each  value  in  the  data  file. 
Additionally,  if  the  field  represents  an  item  used  in  IRT  scaling,  the 
codebook  lists  the  parameters  used  by  the  scaling  program. 

The  frequency  counts  are  not  available  on  the  catalog  file,  but  must  be 
generated  from  the  data  itself.     The  GENFREQ  program  created  the  COUNTS  file 
using  the  field  name  to  locate  the  variable  in  the  database,  and  the  foil 
values  to  validate  the  range  of  data  values  for  each  field.     This  program 
also  serves  as  a  check  on  the  completeness  of  the  foils  in  the  CATALOG  file, 
as  it  flags  any  data  values  not  represented  by  a  foil  value  and  label. 

The  IRT  parameter  file  is  linked  to  the  CATALOG  file  through  the  field 
name.     Printing  of  the  IRT  parameters  is  governed  by  a  control  flag  in  the 
classification  section  of  the  CATALOG  record. 

The  LAYOUT  and  CODEBOOK  files  are  written  by  their  respective  generation 
programs  to  print- image  disk  data  files.     Draft  copies  are  printed  and 
distributed  for  review  before  the  production  copy  is  generated.  The 
production  copy  is  printed  on  an  IBM  ?S00  printing  subsystem  using  laser- 
imaging  technology.     The  printing  is  performed  at  15  characters  per 
horizontal  inch  (pitch)  and  8  lines  per  vertical  inch.     This  accommodates 
printing  of  120  characters  per  line  and  80  lines  per  page  on  standard  8  1/2" 
X  11"  paper. 


6.7.7     SAS  AND  SPSS-X  CONTROL  FILES 

The  SAS  and  SPSS-X  control  statement  files  are  provided  to  the  user  as  a 
means  for  converting  the  raw  data  files  directly  into  a  system  file  for 
subsequent  analyses  under  either  package.     The  files  are  very  similar  in 
their  content  and  structure,  although  actual  implementation  of  their  features 
differ  slightly.     Two  separate  programs,  GENSAS  and  GENSPX,  generate  the 
control  files  using  the  CATALOG  file  as  input. 

Each  of  the  control  files  contain  separate  sections  for  variable 
definition,  variable  labeling,  missing  value  declaration,  value  labeling,  and 
creation  of  scored  variables  from  the  cognitive  items.     The  variable 
definition  section  describes  the  locations  of  the  fields,  by  name,  in  the 
file,  and,  if  applicable,  the  number  of  decimal  places  or  type  of  data.  The 
variable  label  identifies  each  field  with  a  50-character  description.  The 
missing  value  section  declares  which  values  of  which  variables  are  to  be 
treated  as  missing  and  excluded  from  analyses.     The  value  labels  correspond 
to  the  foils  in  the  CATALOG  file.     The  code  values  and  their  descriptors  are 
listed  for  each  discrete  numeric  variable.    The  scoring  section  is  provided 
to  permit  the  user  to  generate  item  score  variables  in  addition  to  the  item 
response  variables. 

Each  of  the  code  generation  programs  combine  three  steps  into  one 
complex  procedure.     As  each  CATALOG  file  record  is  read,  it  is  broken  into 
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several  component  records  according  to  the  information  to  be  used  in  each  of 
the  resultant  sections.     These  record  fragments  are  ^?.gged  with  the  field 
f-quence  number  and  a  section  sequence  code.     They  are  then  sorted  by  secfion 
code  and  sequence  number.     Finally,  the  reorganized  information  is  output  in 
a  structured  format  dictated  by  the  syntax  of  the  processing  language. 

The  generation  of  the  system  files  accomplishes  the  testing  of  these 
control  statement  files.     These  files  are  saved  for  use  by  internal  ETS  users 
of  the  NAEP  data. 


6.7.8    MACHINE -READABLE  CATALOG  FILES 

For  those  NAEP  data  users  who  have  neither  SAS  nor  SPSS-X,  yet  require 
processing  control  information  in  a  computer- readable  format,  the 
distribution  tape  also  contains  machine -readable  catalog  (CAT)  files. 
In  addition  to  processing  control  information,  each  CAT  record  contains  the 
IRT  parameters  and  the  foil  codes  and  labels. 
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Overview  of  Part  II: 
The  Analysis  of  the  1986  NAEP 


Chapter  7 

OVERVIEW  OF  PART  II:     THE  ANALYSIS  OF  1986  NAEP 


Albert  E.  Beaton 
Educational  Testing  Service 


This  chapter  introduces  the  second  part  of  this  technical  report  by 
presenting  an  overview  of  the  procedures  used  in  analyzing  the  1986  NAEP 
data.     The  details  of  the  analytic  procedures  are  contained  in  the  chapters 
that  follow.     The  results  of  these  analyses  are  presented  in  many  substantive 
reports  which  have  been  published  or  are  in  preparation. 

Part  II  of  the  technical  report  assumes  the  existence  of  a  carefully 
edited  database.     The  reader  should  consult  Chapter  1  of  this  report  for 
general  information  about  the  design  of  the  1986  assessment  and  for  an 
overview  of  the  processes  that  went  into  the  construction  of  the  NAEP 
database,   including  the  development  of  objectives  and  items,   the  sampling 
design,  the  measuring  instruments,   field  administration,  professional 
scoring,  data  processing,  and  quality  control.     Detailed  information  about 
these  topics  is  given  in  the  remaining  chapters  of  Part  I. 

Many  of  the  analytical  procedures  used  in  the  1986  NAEP  are  the  same  or 
similar  to  those  used  in  the  1984  assessment,  and  their  details  were  reported 

Implementing  the  New  Design:     The  NAEP  1983-84  Technical  Report  (Beaton, 
1987a).     We  will  not  repeat  general  expository  information  here,  but  refer'to 
that  report  instead.     In  the  following  chapters  describing  subject  area  data 
analyses,   emphasis  will  be  placed  on  how  the  techniques  were  applied. 

The  1986  data  analysis  has  introduced  some  innovations  in  scaling  which 
were  first  introduced  to  NAEP  in  the  1984  reading  and  writing  assessments. 
The  purpose  of  scaling,  as  used  in  NAEP,   is  to  communicate  to  educational 
policymakers  and  the  concerned  public  what  students  in  American  schools  know 
and  can  do.     Using  scaling  and  statistical  estimation  t-chniques,   the  vast 
amount  of  data  collected  in  each  assessed  subject  area  can  be  reduced  to  a 
few,  informative  summary  estimates  of  student  performance. 

The  scaling  process  permits  us  to  take  advantage  of  certain  patterns  of 
responses  in  the  data  in  order  to  reduce  the  information  about  a  student  in 
the  sample  from  his  or  her  responses  to  many  individual  items  to  one  or  a  few 
summary  numbers  that  represent  his  or  her  performance  in  a  subject  area. 
Under  certain  assumptions,   the  scaling  process  allows  us  to  project  the 
performance  of  different  students  onto  the  same  scale  although,   in  NAEP, 
different  students  may  have  been  asked  to  respond  to  different  *  sets  of  items. 
Since  students  who  were  measured  using  different  sets  of  items  will  be 
measured  with  different  precision,  we  have  developed  the  plausible  values 
technology  to  make  consistent  population  estimates  and  appropriate  estimates 
of  their  sampling  errors. 
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Since  the  scaling  process,  as  used  in  NAEP,  is  used  for  data  reduction, 
a  question  arises  as  to  how  far  the  reduction  should  go.     Scaling  uses  some, 
not  all,  of  the  available  information,  and  the  more  the  data  are  reduced,  the 
more  the  detailed  information  in  the  full  database  is  unused  and  unreported. 
In  analyzing  the  1984  assessment  data,  the  decision  was  made  to  reduce  the 
reading  data  to  one  scale  to  represent  a  generalized  reading  proficiency. 
The  studies  of  dimensionality  that  supported  this  decision  are  described  by 
Zwick  (1987).     Reducing  the  data  to  one  overall  scale  does  not  seem  adequate 
for  reporting  results  in  the  areas  of  mathematics  and  science. 

A  major  technical  innova. 'on,  which  was  introduced  in  the  analysis  of 
the  1986  NAEP  data,  was  multivariable  scaling.     In  analyzing  the  1986  data, 
we  decided  to  perform  less  data  reduction  in  mathematics  and  science  by 
developing  and  reporting  several  subscales.     The  subscales  were  determined 
after  reviewing  the  objectives  for  the  subject  areas  and  the  number  of  items 
assessing  those  objectives.     For  mathematics,  the  subscales  were: 

-  Measurement 

-  Numbers  and  operations:  Higher-level 

-  Numbers  and  operations:     Knowledge  and  skills 

-  Geometry 

-  Relations  and  functions 

The  first  three  subscales  span  all  three  grade /age  levels.     Since  the  last 
two  subareas  were  not  assessed  at  all  grade/age  levels,  the  geometry  subscale 
spans  only  grade  7/age  13  and  grade  11/age  17  and  the  relations  and  functions 
is  available  for  grade  11/age  17  only. 

For  science,  the  following  subscales  were  defined: 

-  Life  science 

-  Chemistry 

-  Nature  of  science 

-  Physics 

-  Earth  and  space  science 

-  Physical  science 

The  life  science  and  nature  of  science  subscales  span  all  three  grade/age 
levels.     Since  a  sufficient  number  of  items  was  not  administered  for 
definition  of  all  subscales  at  all  age/grade  levels,  the  chemistry,  physics, 
and  earth  and  space  science  subscales  span  grade  7/age  13  and  grade  11/age  17 
only.     The  physical  science  subscale  is  defined  for  grade  3/age  9  only. 

Although  some  persons  may  prefer  and  make  use  of  the  more  detailed 
information  available  in  the  mathematics  and  science  subscales ,  we  did  not 
wish  to  forgo  a  single  overall  measure  of  performance  in  each  subject  area. 
We  have  assumed  that  other  readers,  particularly  educational  policymakers  at 
the  national  level  as  well  as  the  concerned  public,  would  prefer  and  make  use 
of  a  general  measure  of  overall  proficiency  in  mathematics  or  science, 
although  it  is  possible  that  such  an  overall  scale  might  mask  gains  in  some 
subscales  that  are  balanced  by  losses  in  others.    We  therefore  also  developed 
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composites  of  the  subscales  to  represent  overall  proficiency  in  mathematics 
and  science.     The  reader  or  the  user  of  the  public -use  data  tapes  may, 
therefore,  focus  on  either  the  overall  scales,  the  subscales,  or  both! 

The  overall  estimate  of  a  student's  proficiency  in  mathematics  or 
science  is  a  weighted  average  of  his  or  her  proficiency  estimates  for  the 
several  subscales  in  the  area.     The  weights  used  in  foDiiing  the  overall 
estimate  was  derived  from  the  importance  that  the  Learning  Area  Committee  in 
the  subject  area  placed  on  that  objective. 

The  1984  reading  assessment  introduced  scale  anchoring  as  a  way  of 
communicating  to  educational  policymakers  what  students  in  American  schools 
know  and  can  do.     We  wished  to  anchor  the  mathematics  and  science  scales  as 
well,  but  did  not  feel  that  enough  items  were  available  to  anchor  at  the 
subscale  level,  and  so  decided  to  anchor  only  the  overall  scales.     Since  the 
overall  scales  were  not  derived  directly  from  item  response  theoretic 
methods,  the  method  used  in  1984  and  described  by  Beaton  (1987b),  which  uses 
IRT  item  parameters,  was  not  directly  applicable.     Thus,  a  different  way  of 
scale  anchoring  had  to  be  used. 

As  mentioned  in  Implementing  the  New  Design,   scale  anchoring  is  not 
dependent  on  the  parameters  of  a  model  such  as  the  three-parameter  logistic 
model,  which  is  used  in  item  response  theory.     The  basic  idea  of  scale 
anchoring  is  locating  items  that  discriminate  between  selected  points  on  a 
scale,  and  then  analyzing  those  items  to  describe  what  it  is  that  students  at 
a  higher  level  can  do  that  students  at  a  lower  level  cannot.     The  location  of 
discriminating  items  was  done  directly,  that  is,  by  computing  the  proportion 
of  students  at  or  near  each  anchor  point  on  the  overall  scale  who  responded 
correctly  to  an  item  and  then  comparing  that  proportion  with  the  proportion 
of  students  at  the  next  lower  anchor  point  who  correctly  responded.  Items 
with  large  increases  in  proportion  correct  at  particular  scale  points  were 
referred  to    subject  matter  experts  for  interpretation.     Both  the  overall 
mathematics  and  science  composites  were  anchored  in  this  way. 

A  substantial  part  of  the  1986  NAEP  data  analysis  involved  the  reading 
anomaly.     The  1986  NAEP  trend  data  indicated  a  substantial  drop  in  the 
average  reading  proficiency  of  students  at  the  9-  and  17-year-old  levels  and 
increases  in  variance  at  all  three  age  levels.     The  changes  were  so 
substantial  as  to  be  deemed  anomalous,  and  a  major  effort  was  made  to  explain 
whac  happened.     The  NAEP  staff  examined  a  number  of  hypotheses  about  what 
might  have  caused  such  anomalous  results,   including  changes  in  the  student 
population,  modifications  of  the  NAEP  design  and  administrative  procedures, 
lapses  in  quality  control,  computer  bugs,  as  well  as  external  uncontrollable 
events.     Although  the  results  of  some  of  its  investigations  are  inconclusive, 
some  hypotheses,   such  as  inaccuracies  in  sampling,   scaling,  and  quality 
control,  can  be  ruled  out  beyond  a  reasonable  doubt.     The  design  of  the  1988 
NAEP  was  modified  to  explore  the  reading  anomaly  further.     The  study  of  the 
reading  anomaly  is  presented  in  another  report  :.he  NAEP  1985-86  Reading 
Anomaly:  A  Technical  Report  (Beaton,  1988). 
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The  remaining  chapters  in  this  part  of  the  report  are  as  follows: 

Chapter  8  is  a  general  discussion  of  the  scaling  procedures  used  in 
NAEP,  including  the  multivariable  scaling  procedures  used  for  mathematics  and 
science  and  the  WARM  method  that  was  used  for  background  and  attitude 
questions.     This  chapter  also  discusses  the  procedures  that  created  the 
plausible  values  and  suggestion  about  how  these  values  can  be  used  for 
consistent  estimates  of  population  parameters  and  their  standard  errors. 
Sources  of  bias  in  secondary  analyses  are  also  discussed.     Finally,  this 
chapter  presents  an  overview  of  the  1986  NAEP  scales  and  the  conditioning 
variables  used  in  their  creation. 

Chapter  9  presents  the  details  of  the  analysis  procedures  used  with  the 
1986  reading  data.     The  reading  data  were  scaled  using  the  same  scaling 
method  that  was  used  in  1984,  which  produced  a  single  overall  reading  scale. 
However,  the  trend  results  were  deemed  anomalous,  and  have  not  been  reported 
to  the  public.     Cross -sectional  analyses  were  performed  and  reported  in  Who 
Reads  Best?    Factors  Related  to  Reading  Achievement  in  Grades  3,  7,  and  11 
(Applebee,  Langer,  6cMullis,  1988). 

Chapter  10  presents  the  details  of  the  analysis  of  the  mathematics  data. 
For  developing  trend  information,  the  1986  design  included  subsamples  in 
which  mathematics  was  assessed  in  the  same  way  as  in  the  1972-73,  1977-78, 
and  1981-82  assessments,  the  only  years  in  which  mathematics  was  assessed  in 
the  past.     The  number  of  items  common  to  all  assessments  in  the  different 
years  was  small,  and  so  a  single  overall  mathematics  scale  was  developed  for 
trend  reporting.     The  main  NAEP  sample  in  the  1986  assessment  had  a 
sufficient  number  of  mathematics  items  for  multivariable  scaling,  and  so 
proficiencies  on  several  subscales  were  estimated,  and  the  estimates  were 
used  for  cross -sectional  analyses.     The  results  of  the  mathematics 
assessment,  both  trend  and  cross-sectional,  are  presented  in  The  Mathematics 
Report  Card:     Are  We  Measuring:  Up?    Trends  and  Achievement  Based  on  the  1986 
National  Assessment  (Dossey,  Muliis,  Lindquist,  &  Chambers,  1988). 

Chapter  11  presents  the  details  of  the  science  data  analyses.  The 
general  form  of  the  science  analyses  was  the  same  as  matheiratics ,  since  this 
was  also  the  first  time  science  was  scaled.     Science  was  assessed  in  the  past 
in  1971-72,  1977-78,  and  1981-82.     The  results  of  the  science  analyses  will 
be  published  in  August,  1988. 

Chapter  12  presents  the  methodology  used  in  the  computer  competence 
assessment.     The  1986  assessment  was  the  first  to  include  computer  competence 
and  thus  there  was  no  trend  information  to  report.     Since  the  types  of 
information  was  varied  and  its  structure  unexplored,  no  scaling  was  attempted 
in  this  area.     The  statistical  methodology  was  limited  to  the  average 
percentage  correct  method  that  was  used  in  past  assessments.     The  computer 
competence  results  are  presented  in  Computer  Competence:     The  First  National 
Assessment  (Martinez  &  Mead,   1988)  . 

Chapter  13  describes  the  analytic  procedures  used  in  the  analysis  of  the 
U.S.  history  and  literature  assessments.     U.S.  history  and  literature  were 
assessed  at  grade  11/age  17  only.     Neither  subject  area  had  comparable  data 
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available  from  past  assessments,  and  so  no  trend  analysis  was  possible.  The 
dimensionality  of  the  two  subject  areas  was  studied,  and  one  overall  scale 
for  each  area  was  decided  upon.     The  results  of  the  scaling  and  analysis  have 
been  published  in  the  NAEP  report  Literature  &  U.S.  History:  The 
Instructional  Experience  and  Factual  Knowledge  of  Hiph  School  Juniors 
(Applebee,  Langer,  &  Mullis ,  1987), 

Chapter  14  presents  the  analytical  procedures  used  in  developing  the 
sampling  weights,   including  various  nonresponse  adjustments,  trimming,  and 
poststratification.     This  chapter  also  details  the  jackknife  procedure  used 
to  make  estimates  of  sampling  variance  in  the  NAEP  analyses.     The  final 
section  shows  estimates  of  the  design  effects  for  1986  NAEP. 


1  o  ^ 
-173-      lb  4 


CHAPTER  8 
Scaling  Procedures 


163 


Chapter  8 
SCALING  PROCEDURES''" 


Robert  J.  Mislevy 
Educational  Testing  Service 


8 . 1  INTRODUCTION 


A  key  innovation  in  NAEP  during  the  ETS  tenure  is  scale-score  reporting. 
With  scale-score  methods,  the  performance  of  a  sample  of  students  in  a 
subject  area  or  subarea  can  be  summarized  on  a  single  scale  even  if  different 
students  have  been  administered  different  exercises.     Similar  procedures  can 
be  used  to  summarize  responses  to  sets  of  related  background  questions.  This 
chapter  presents  an  overview  of  the  scaling  methodologies  employed  in  the 
analyses  of  the  1986  NAEP  surveys: 

Section  8.2  briefly  discusses  the  perspective  on  scaling 
from  which  the  procedures  were  conceived  and  applied. 

Section  8.3  reviews  the  "plausible  values"  methodology 
used  in  NAEP  scale- score  analyses. 

Section  8.4  describes  how  plausible  values  are  used  in 
subsequent  analyses. 

Section  8.5  lists  the  scale-score  analyses  carried  out  on 
the  1986  data. 

Details  of  scaling  procedures  for  specific  subject  areas  are  presented  in 
Chapters  9,   10,  11,  and  13. 

8.2    SCALING  IN  NAEP 

As  it  was  originally  conceived  some  twenty  years  ago,  NAEP  was  intended 
to  limit  reports  to  percents-correct  on  individual  items.     It  soon  became 
apparent,  however,   that  some  level  of  aggregation  was  needed  to  communicate 
results  effectively.     Average  percents-correct  over  sets  of  items,  as 
introduced  by  the  Education  Commission  of  the  States,   improved  reporting  by 
providing  such  an  aggregation.     Their  limitation  to  specific  and  unchangeable 
sets  of  items  hampered  the  refinement  of  the  item  pool  over  time,  however, 
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and  provided  no  information  about  the  distributions  of  skills  among  students 
in  targeted  populations. 


These  limitations  can  be  overcome  by  the  use  of  response  scaling 
methods.     If  several  items  require  similar  skills,   the  regularities  observed 
in  response  patterns  can  often  be  exploited  to  characterize  both  respondents 
and  items  in  terms  of  a  relatively  small  number  of  variables.     \fJhen  combined 
through  appropriate  mathematical  formulas,   these  variables  capture  the 
dominant  features  of  the  data.     Using  the  scale,   it  becomes  possible  to  talk 
about  distributions  of  proficiency  in  a  population  or  subpopulation,  and  to 
estimate  the  relationships  between  proficiency  and  background  variables. 

Early  work  on  scaling  is  attributed  to  Thurstone,  but  the  more  recent 
development  of  item  response  theory  (IRT;  e.g..  Lord,  1980)  has  been 
particularly  influential  on  measurement  practice.     IRT  and  a  newly  developed 
procedure  called  the  weighted  average  response  method  (WARM) ,  both  of  which 
are  reviewed  in  section  8.3,  are  the  two  scaling  procedures  ETS  has 
introduced  in  NAEP  reporting. 

We  hasten  to  point  out  that  any  procedure  of  aggregation,   from  a  simple 
average  to  a  complex  multidiiiiensional  scaling  model,  highlights  certain 
patterns  by  collapsing  over  others.     In  a  very  real  sense,  every  single  item 
in  a  NAEP  survey  is  of  interest  in  its  own  right,   and  may  provide  useful 
information  about  what  young  Americans  know  and  can  do.     The  choice  of  an 
aggregation  procedure  must  be  driven  by  a  conception  of  just  which  patterns 
are  salient  for  a  particular  purpose.     The  procedure  that  is  optimal  for  one 
purpose  may  be  poorly  suited  for  another. 

The  relatively  high  levels  of  aggregation  found  in  ETS/NAEP  reports  such 
as  The  Reading  Report  Card:     Progress  Toward  Excellence  in  Our  Schools 
(1985),   for  example,  are  well  suited  to  high-level  discussions  of  trends  and 
policy  implications.     They  average  over,   and  therefore  are  not  keyed  to,  the 
microanalysis  of  performance  at  the  level  of  specific  skills,  as  might  be 
desired  by  educational  psychologists;   they  do  not  reveal  popular  student 
misconceptions  or  erroneous  rules,  as  might  be  of  interest  to  classroom 
teachers  in  a  subject  area.     For  the  first  of  these  latter  purposes,  one 
might  prefer  the  precision  of  a  latent  class  model  for  more  highly  specified 
skills.     For  the  second,  detailed  discussions  of  results  for  individual  items 
might  be  more  appropriate.     By  no  means  do  the  scale-score  methods  we  er^ploy 
as  a  reporting  vehicle  exhaust  the  potential  of  NAEP  data;  neither  do  they 
preclude  other  researchers  from  carrying  out  alternative  analyses  from 
different  perspectives. 

A  reporting  scale  in  the  1986  NAEP  survey  simply  provides  a  summary  of 
performance  on  a  collection  of  items.     To  be  useful  in  the  context  of 
educational  assessment,  such  collections  must  be  defined  in  terms  of  (1)  the 
psychology  of  school  learning  (Messick,   1984),  since  this  reflects  how  pupils 
learn,  and  (2)  the  organization  of  schools,  since  this  is  the  channel  through 
which  educational  policy  flows.     The  following  paragraphs  outline  the 
perspective  on  scale  delineation  employed  in  the  1986  assessment,  as  based  on 
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the  experiences  of  not  only  NAEP  itself,  but  of  the  California  Assessment 
Program  (CAP)  and  the  assessments  of  Britain's  National  Foundation  for 
Educational  Research  (NFER) . 

CAP  and  NFER  offer  two  early  experiences  concerning  the  breadth  of 
scaling,  both  occurring  during  the  mid-  to  late  seventies.     NFER  employed  a 
collection  of  broadly  defined  scales  in  conjunction  with  an  IRT  model  that 
made  strong  assumptions  about  item  reliabilities  and  the  character  of  change. 
Unfortunately,  this  approach  led  to  measures  whose  meanings  could  not  always* 
be  maintained  over  time  or  from  one  subpopulation  to  another  (Goldstein, 

1980)  .     Partly  in  response  to  these  problems,  CAP  took  an  opposite  tack' 
delineating  large  numbers  of  narrowly  defined  scales  within  each  subject  area 
at  each  assessment  grade  level.     Third- grade  reading,  for  example,  comprises 
seventeen    indivisible  skill  elements,"  each  conceived  so  that  changes  in 
curricular  emphasis  have  similar  impact  on  all  its  items  (Bock  &  Mislevy, 

1981)  .     The  adequacy  of  this  approach  has  been  borne  out  empirically  by  the 
stability  of  item  parameter  estimates  over  time  (Mislevy  &  Bock,  1982). 

The  specificity  of  the  CAP  scales  offers  the  possibility  of  detecting 
the  effects  of  small  shifts  in  curricular  emphases,  an  important  concern  for 
an  assessment  whose  primary  focus  is  the  school.     The  CAP  scales  prove  more 
specific  than  necessary  for  tracking  more  broadly  conceived  trends  in 
performance.     After  correction  for  measurement  error,  CAP  school -level 
correlation  matrices  among  elements  exhibit  blocks  of  entries  that  are 
essentially  unity.  Parallel  patterns  of  trends  within  such  blocks  of  elements 
also  indicate  that  the  integrity  of  scales  within  reading,  for  instance, 
would  be  maintained  if  they  were  defined  at  the  level  of  "inferential 
comprehension"  rather  than  of  five  subcategorizations  within  inferential 
comprehension. 

The  evidence  on  scaling  for  educational  assessment,  then,  suggests  that 
one  scale  for  a  subject  area  is  probably  too  few,  but  twenty  is  probably  too 
many.     ETS/NAEP  scaling  procedures  for  mathematics  and  science  reflect  this 
view.     Five  scales  in  mathematics  and  six  in  science,  along  with  a  composite 
in  each  area,  have  been  designed  to  capture  the  essential  subdivisions  of  the 
subject  areas  as  indicated  by  NAEP's  learning  area  committees.      We  initially 
fit  models  to  data  along  these  lines,  and  modified  the  final  analyses  when 
unacceptable  model  violations  resulted  (see  chapters  on  specific  subject 
areas) . 

Scaling  within  subareas  does  not  ignore  the  desire  of  policymakers  and 
the  public  at  large  for  a  single  index  of  performance  in  a  subject  area- -one 
of  the  objectives  of  an  educational  assessment  discussed  by  Bock  and  Mislevy 
(1988)  in  their  introduction  of  the  "duplex  design."    A  summary  is  easily 
obtained  as  the  (possibly  weighted)  average  of  results  across  subscales .  The 
resulting  composite  scores  are  useful  to  policymakers  as  quick  summaries  of 
overall  performance,  and  to  secondary  researchers  interested  in  the 
relationships  between  performance  and  student  background  variables.  Compared 
to  fitting  a  single  scale  to  a  broadly  defined  subject  area,  though,  the 
approach  of  first  scaling  within  narrowly  defined  subareas  more  closely 
satisfies  the  assumptions  of  scaling  models  and  maintains  the  capability  to 
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characterize  important  interactions  or  countertrends  within  educationally 
relevant  subdivisions  of  the  subject  area  as  a  whole. 

8.3     NAEP  SCALING  METHODOLOGY 

The  paragraphs  that  follow  review  the  scaling  models  employed  in  the 
analyses  of  1986  NAEP  data,  and  the  "plausible  values"  methodology  that 
allows  such  models  to  be  used  with  NAEP's  sparse  item- sampling  design.  The 
reader  is  referred  to  Mislevy  (1988)  for  an  introduction  to  plausible  values 
methods  and  a  comparison  with  standard  psychometric  analyses,  to  Mislevy  and 
Sheehan  (1987)  and  Beaton  and  Johnson  (1987)  for  additional  information  on 
how  the  models  are  used  in  NAEP,  and  to  Rubin  (1987)  for  the  theoretical 
underpinnings  of  the  approach. 

8.3.1    The  Scaling  Models 

Two  types  of  scaling  models  were  used  by  NAEP  in  the  1986  assessment. 
For  the  subject  areas,  the  3-parameter  logistic  (3PL)  model  from  item 
response  theory  (IRT;  e.g.,  Lord,   1980)  was  used.     For  selected  sets  of 
background  questions,  the  weighted  average  response  method  (WARM;  Beaton  & 
Johnson,  1987),  developed  by  NAEP  for  the  1984  assessment,  was  used.  Both 
are  "latent  variable"  models,  quantifying  respondents'  tendencies  to  provide 
responses  in  a  given  direction  (e.g.,  correct  answers  to  items  in  a  subject 
area;  positive  responses  on  attitude  questions),  as  a  function  of  a  parameter 
that  is  not  directly  observed. 

The  3-parameter  logistic  (3PL)  IRT  model.  The  fundamental  equation  of 
the  3PL  is  the  probability  that  a  person  whose  proficiency  is  characterized 
by  the  unobservable  variable  6  will  respond  correctly  to  item  j : 

P(x^l|$,a    b    c  )  =  c    +  (1-c  )/{l+exp[-1.7a  ($-b  )]} 

^  Pj(^)     .  (8.1) 

where 

is  the  response  to  item  j ,  1  if  correct  and  0  if  not; 

a.  ,     where  a.>0,  is  the  slope  parameter  of  item  j,  characterizing  its 

sensiti"i^ity  to  proficiency; 

b.  is  the  threshold  parameter  of  item  j,  characterizing  its 
difficulty;  and 

c.  where  0<c.<l,   is  the  lower  asymptote  parameter  of  item  j, 
reflectin;^.  the  chances  of  a  correct  response  from  students  of  very 
low  proficiency.     In  1986  NAEP  analyses,  c  parameters  were 
estimated  for  multiple-choice  items,  but  were  fixed  at  zero  for 
open-ended  items. 
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For  the  purposes  of  reporting  item  parameter  estimates  and  other 
intermediary  estimates,  the  linear  indeterminacy  apparent  in  (8.1)  may  be 
resolved  by  an  arbitrary  choice  of  the  origin  and  unit  size  in  a  given  scale. 
This  was  done  for  the  reading  scale  in  1984  by  standardizing  the  combined 
grade  4/age  9,  grade  8/age  13,  and  grade  11/age  17  samples.    To  aid 
interpretation,  final  published  results  are  reported  on  scales  that  are 
transformed  linearly  .from  the  d  scale  in  ways  related  to  the  0- to -500  reading 
proficiency  (RP)  scale  developed  in  the  1984  NAEP  assessment  of  reading 
(Beaton,  1987b).    These  transformations  are  described  in  the  appropriate 
subject  area  chapters  in  this  report. 

Under  the  usual  IRT  assumption  of  local  independence,  the  probability  of 

a  vector  x-(x-,  x^)  of  responses  to  n  items  is  simply  the  product  of  terms 

based  on  (8.1;: 


n  X.  1-x. 

P(x|^a,b,c)  =n  [P,(^?)]  J[l-P.(^?)]      J   .  (8.2) 


After  X  has  been  observed,   (8.2)  can  be  considered  a  likelihood  function,  and 
provides  a  basis  for  inference  about  6  or  about  item  parameters.     In  NAEP, 
estimates  of  item  parameters  were  obtained  with  Mislevy  and  Bock's  (1982)' 
BILOG  computer  program,   then  treated  as  known  in  subsequent  calculations. 
Once  items  have  been  calibrated  in  this  manner,  a  likelihood  function  for  $ 
is  induced  by  a  ve:itor  of  responses  to  any  subset  of  calibrated  items,  thus 
allowing  ^ -based  inference  from  matrix  samples. 

In  all  NAEP  IRT  analyses,  missing  responses  at  the  end  of  each  block  a 
student  was  presented  were  considered  as  not-reached,  and  treated  as  if  they 
had  not  been  presented.     Missing  responses  before  the  last  observed  response 
in  a  block  were  considered  intentional  omissions,  and  treated  as  fractionally 
correct  at  the  value  of  the  reciprocal  of  the  number  of  response 
alternatives.     These  conventions  are  discussed  by  Mislevy  and  Wu  (1988). 
With  regard  to  the  handling  of  not-reached  items,  they  find  that  ignoring 
not- reached  items  introduces  slight  biases  into  item  parameter  estimation  to 
the  degree  that  (i)  not- reached  items  are  present  and  (ii)  speed  is 
correlated  with  ability.     With  regard  to  omissions,  they  find  that  the  method 
described  above  provides  consistent,  though  limited  information,  estimates  of 
item  and  ability  likelihoods  under  the  assumption  that  respondents  are  acting 
in  accordance  with  directions  to  omit  only  if  they  can  do  no  better  than 
responding  randomly. 

The  weighted  average  response  method  (WARM^ .     The  basic  equation  of  the 
WARM  is  an  average  of  item  responses: 


-  w'x  .  (8.3) 
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Here  w  is  a  vector  of  constants,  specified  so  as  to  provide  a  meaningful 
summary  of  performance.    Weights  of  1/n  for  an  n-item^^est,  for  example, 
yield  simply  an  average  score;  weighs  given  by  the  k      eigenvector  of  the 
covariance  matrix  for  x  yield  the  k      component  score.     If  a  respondent 
responded  to  all  items,  then  a  WARM  score  would  be  directly  calculable  via 
(8.3)  without  error.     Typically,  however,  a  given  NAEP  respondent  receives 
only  a  subset  of  the  items  in  a  WARM  scale,  so  that  his  or  her  WARM  6  is  not 
observed  directly. 

8.3.2    An  Overview  of  Plausible  Values  Methodology 

Item  response  theory  was  developed  in  context  of  measuring  individual 
examinees'  abilities.     In  that  setting,  each  individual  is  administered 
enough  items  (often  100  or  more)  to  permit  precise  estimation  of  his  or  her 

A 

^ ,  as  a  maximum  likelihood  estimate  6,  for  example.     Because  the  uncertainty 
associated  with  each  6  is  negligible,  the  distribution  of  ^,  or  the  joint 
distribution  of  6  with  other  variables,  can  then  be  approximated  using 

individuals'   6  values  as  if  they  were  d  values. 

This  approach  breaks  down  in  the  assessment  setting  when  each  respondent 
is  administered  fewer  items  in  any  single  scaled  area,   in  order  to  provide 
broader  content  coverage  in  limited  testing  times.     (It  was  in  fact  attempted 
by  ETS  in  early,  aborted  analyses  of  the  1984  NAEP  reading  survey;  see 
Beaton,   1987,  section  10.2.)     The  main  difficulty  is  that  the  uncertainty 
associated  with  individual  Ss  is  too  large  too  ignore,  and  the  features  of 

the  6  distribution  can  be  seriously  biased  as  estimates  of  the  6 
distribution.     Plausible  values  were  developed  as  a  means  of  obtaining 
consistent  estimates  of  at  least  some  population  features,  and  approximations 
of  others  no  worse  than  would  be  obtained  using  standard  IRT  procedures.  A 
detailed  development  of  plausible  values  methodology  is  given  in  Mislevy 
(1988).     Along  with  theoretical  justifications,  that  paper  presents 
comparisons  with  standard  procedures,  discussions  of  biases  that  arise  in 
some  secondary  analyses,  and  numerical  examples.     The  following  paragraphs 
give  a  brief  overview  of  the  approach,  focusing  on  its  implementation  in  the 
1986  NAEP  analyses. 

Let  y  represent  the  responses  of  all  sampled  examinees  to  background  and 
attitude  questions.     If  IRT  or  WARM  6  values  were  available  for  all  sampled 
examinees,   it  would  be  possible  to  compute  a  statistic  t(i,y)--such  as  a 
subpopulation  sample  mean,  a    sample  percentile  point,  or  a  sample 
variance- -to  estimate  a  corresponding  population  quantity  T.     A  function 
Ud,y)--e.g.,  a  jackknife  estimate- -would  be  used  to  gauge  sampling 
uncertainty. 

Because  the  3PL  and  the  WARM  are  latent  variable  models,  however,  d 
values  are  not  observed  even  for  sampled  students.     In  the  U.S.  history  and 
literature  assessments,  where  enough  responses  are  solicited  from  each 

A 

Student  to  provide  a  fairly  precise  estimate  6  of  his  or  her  6  value,  values 

A  A 

of  and  U(i,y)  are  reported  as  approximations  of  corresponding  t(i,y_) 
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and  U(i,i)  values.     In  other  subject  areas,  and  with  WARM  background- 
variable  scales,  the  small  numbers  of  items  administered  to  most  sampled 
students  preclude  this  simple  approximation.     In  these  areas,  "plausible 
values"  methods  were  used. 


Following  Rubin  (1987),  we  can  think  of  i  as  "missing  dat i"  and 
approximate  t(i,^)  by  its  expectation  given  (x,:^) ,  the  data  that  actually 
were  observed,  as  follows: 


/  t(i,:£)  p(i|x,3r)  dl    .  (8.4) 


It  is  possible  to  approximate  t    using  random  draws  from  the  conditional 
distributions  p(^|x    y^)  of  each  sampled  student  i.     These  values  are 
referred  to  as  imputations  in  the  sampling  literature,  and  "plausible  values" 
in  NAEP.     The  value  of  $  for  any  respondent  that  would  enter  into  the 
computation  of  t  is  thus  replaced  by  a  randomly  selected  value  from  the 
conditional  distribution  for  $  given  his  or  her  responses  to  cognitive  items 
(x^)  and  background  items  (y^) .     Rubin  (1987)  proposes  this  process  be 
carried  out  several  times- -multiple  imputations- -so  that  the  uncertainty 
associated  with  imputation  can  be  quantified.     The  average  of  the  results  of 
K  estimates  of  t,  each  computed  from  a  different  set  of  plausible  values,  is 
a  Monte  Carlo  approximation  of  (8.4);  the  variance  among  them,  B,  reflects 
uncertainty  due  to  not  observing  8,  and  must  be  added  to  the  estimated 
expectation  of  U(i,x) ,  which  reflects  uncertainty  due  to  testing  only  a 
sample  of  students  from  the  population.     Section  8.4  explains  how  these 
plausible  values  are  used  in  subsequent  analyses. 

It  cannot  be  emphasized  too  strongly  that  plausible  values  are  not  test 
scores  for  individuals  in  the  usual  sense. 

Plausible  values  are  offered  only  as  intermediary  computations  for 
calculating  integrals  of  the  form  of  Equation  8.4  in  order  to  estimate 
population  characteristics.     VThen  the  underlying  model  is  correctly 
specified,  plausible  values  will  provide  consistent  estimates  of  population 
characteristics,  even  though  they  are  biased  estimates  of  the  proficiencies 
of  the  individuals  with  whom  they  are  associated. 

In  both  IRT  and  WARM  analyses  in  NAEP,  plausible  values  are  included  for 
students  who  were  presented  items  in  a  subject  area  or  subarea,  but  did  not 
respond  to  any  of  them.     The  conditional  distribution  employed  here  for  such 
a  nonrespondent  is  based  solely  on  his  or  her  background  values  y. 
Nonrespondents  were  included  in  this  manner,  even  though  they  provide  no 
information  about  their  proficiency,  in  order  to  maintain  the 
representativeness  of  the  sample.     This  convention  provides  estimates  of 
population  characteristics  that  have  the  same  expected  value  and  precision  as 
would  be  obtained  under  the  more  familiar  nonresponse  adjustment  of  deleting 
the  nonrespondents  and  boosting  the  sampling  weights  of  respondents  with  the 
same  y  values . 
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8.3.3    Computing  Plausible  Values  in  IRT-based  Scales 


Plausible  values  for  each  respondent  i  are  drawn  from  the  conditional 
distribution  p(^|x.,y.).     This  subsection  describes  how,  in  IRT-based  scales, 
these  conditional  distributions  are  characterized  and  how  the  draws  are 
taken.     Using  conditional  independence, 

P(^  |x.,y.)  cc  P(x.l^?)  p(^|y.),  (8.5) 

where  P(x^|^)  is  the  likelihood  function  for  6  induced  by  observing  x. 
(treating  item  parameter  estimates  as  known  true  values)  and  p(^|y.)  is  the 
distribution  of  6  given  the  observed  value  y^  of  background  responses. 

Equations  (8.4)  and  (8.5)  can  also  be  employed  with  vector-valued  $,  as 
in  the  1986  NAEP  mathematics  subscales.     In  such  cases,   P(x.|^)   is  the 
product  over  subscales  of  the  indeperdent  likelihoods  induced  by  the  items 
within  each  subscale,  and  p(^|y.)  is  the  multivariate- -and  generally  non- 
independent-  -j_oijvt__density;  of  proficiencies  for  the  subscales,  conditional  on 
background  variables  y. 

In  the  analyses  of  1986  NAEP  data,  a  normal  (Gaussian)  form  was  assumed 
for  p(^ly^),  with  a  common  dispersion  and  with  a  mean  given  by  a  main-effects 
model  for  selected  elements  of  the  complete  vector  of  background  variables . 
The  included  background  variables  will  be  referred  to  as  the  conditioning 
variables .  and  will  be  denoted  y  .     (The  conditioning  variables  used  in  1986 
NAEP  analyses  are  given  in  Table  8.4.)      The  following  model  was  fit  in  each 
subject  area: 

^  -  r  y^  +  £  ,  (8.6) 

where  €  is  normally  distributed  with  mean  0  and  dispersion  S.     F  and  2  are 
the  parameters  to  be  estimated.     In  subject  areas  with  only  one  scale,  such 
as  reading,  F  is  a  vector  and  S  is  a  scalar.     In  subject  areas  comprising 
subscales,  F  is  a  matrix  and  2  is  a  covariance  matrix.     Like  item  parameter 
estimates,  these  estimates  of  the  parameters  of  conditional  distributions 
were  treated  as  known  true  values  in  subsequent  steps  of  the  analyses. 
(Planned  developments  for  future  assessments  will  take  thi^  uncertainty  into 
account. ) 

Maximum  likelihood  estimates  of  F  and  E  were  obtained  with  Sheehan's 

(1985)  M-GROUP  computer  program,  using  a  variant  of  the  EM  solution  described 

in  Mislevy  (1985).     The  difference  from  the  published  algorithm  lies  in  the 

numerical  approximation  that  was  employed.     Note  from  (8.5)  that  p(^|x.,y.) 

is  proportional  to  the  product  of  two  terms,  the  likelihood  P(x.|^)  ancl  the 

conditional  distribution  p(^|y.).     The  conditional  distribution  has  been 

1  c  c 

assumed  multivariate  normal,  with  mean  /^j^      ^  Yj^         covariance  matrix  2;  if 
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the  likelihood 
and  covariance 

normal  with  covariance  inatrix 


is  approximated  by  another  normal  distribution,  with  mean  yh 
matrix  2,,  then  the  posterior  p(^|x. ,y  )  is  also  multivariate 


SP  -  (S-^  +  (sj;)-^   -1  (8.7) 
and  mean  vector 

h  -  +  e\^\  "^(2^)-^     .  (8.8) 

In  the  1986  analyses,  a  normal  approximation  for  P(x.|^)  is  accomplished 
m  a  given  scale  by  the  steps  described  below.     (Recall  t^iat  by  the  assumed 
conditional  independence  across  scales,  the  joint  conditional  likelihood  for 
multiple  scales  is  the  product  of  independent  likelihoods  for  each  of  the 
scales.)    These  computations  are  carried  out  in  the  scale  determined  by  BILOG 
(Misleyy  &  Bock,   1982)  item  parameter  estimates,  where  the  mean  and  standard 
deviation  of  the  composite  population  formed  by  combining  the  three  NAEP 
grade/ages  has  mean  zero  and  standard  deviation  one. 

1.  Lay  out  a  grid  of  Q  equally  spaced  points  from  -5  -o  +5, 
a  range  that  covers  the  region  in  each  scale  where  all 
examinees  from  all  NAEP  grade/age  groups  are  virtually 
certain  to  occur.     The  value  of  Q  varies  from  20  to  40, 
depending  on  the  scale  being  usedj  smaller  values  suffice 
for  scales  with  few  items  given  to  each  respondent,  while 
larger  values  are  required  for  scales  with  many  items. 

2.  At  each  point  X      compute  the  likelihood  L(x.|^-«X  ) 

^  i'  q 

3.  To  improve  the  normal  approximation  in  those  cases  in 
which  likelihoods  are  not  roughly  S3rmmetric  in  the  range 
of  interest  —  as  when  all  of  a  respondent's  answers  are 
correct—multiply  the  values  from  Step  2  by  the  mild 
smoothing  function 

exp(X  +5) 
S(X  )  -  —  9  


q 


[ l+exp (X^+5 ) ] [ 1+exp (X^ -  5 ) ] 


This  is  equivalent  to  augmenting  each  examinee's  response 
vector  with  responses  to  two  fictitious  items,  one 
extraordinarily  easy  item  that  everyone  gets  right  and  one 
extraordinarily  difficult  item  that  everyone  gets  wrong.  This 
expedient  improves  the  normal  approximation  for  examinees  with 
flat  or  degenerate  likelihoods  in  the  range  where  their 
conditional  distributions  lie,  but  has  negligible  effects  for 
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examinees  with  even  modestly  well -determined  S3rmmetric 
likelihoods . 


4.      Compute  the  mean  and  standard  deviation  of  8  using  the 
weights  S(X^)  obtained  in  Step  3. 

At  this  stage,  then,  the  likelihood  induced  by  a  respondent's  answers  to 
the  items  in  a  given  scale  is  approximated  by  a  normal  distribution.     In  an 
area  such  as  reading  where  there  is  only  one  scale,  a  single  normal 
distribution  thus  summarizes  information  from  item  responses.     In  an  area 
such  as  mathematics  or  science  where  there  are  several  scales,  independent 
normal  distributions,  one  per  subscale,  summarize  information  from  responses 
to  items  from  the  several  scales. 

This  normalized-likelihood/normal  posterior  approximation  was  then 
employed  in  both  the  estimation  of  F  and  S  and  in  the  generation  of  plausible 
values.     From  the  final  estimates  of  F  and  S,  a  respondent's  posterior 
distribution  was  obtained  from  the  normal  approximation  using  the  four -step 
procedure  outlined  above.     A  plausible  value  was  drawn  from  this  normal 
distribution- -univariate  normal,  in  areas  like  reading  with  only  a  single 
scale;  multivariate  normal  in  areas  like  mathematics  and  science,  with 
multiple  subscales.     For  those  subject  areas  with  multiple  subscales, 
weighted- average  composites  over  subscales  were  also  calculated  after 
appropriate  rescaling  (see  subject  area  chapters  for  definitions  of 
composites) . 

8.3.3.2      Computing  Plausible  Values  in  WARM  Scales 

In  1986  NAEP,  the  weighted  average  response  method  (WARM)  was  used  to 
create  composite  variables  from  related  background  questions.     The  creation 
of  the  WARM  composites  proceeded  in  two  steps: 

(1)  the  definition  of  the  composite  variables,  and 

(2)  the  construction  of  student- level  plausible  values  for  each 
composite . 

The  process  of  developing  the  composite  variables  for  the  background 
factors  relating  to  achievement  in  a  given  subject  area  was  initiated  by 
conducting  a  factor  analysis  of  the  results  of  the  pool  jf  background 
questions  related  to  that  subject  area  for  each  of  the  three  grade/age  levels 
separately.     As  was  the  case  for  the  cognitive  questions,  the  background 
questions  were  included  in  the  BIB  spiralled  blocks  in  such  a  way  that  no 
student  responded  to  all  of  the  questions.     However,  since  BIB  spiralling  has 
the  property  that  every  pair  of  questions  is  administered  to  a  randomly 
equivalent  subsample  of  students,  all  intercorrelations  among  the  pool  of 
background  questions  can  be  consistently  estimated  from  the  responses  of  the 
students  answering  each  pair  of  questions.     This  resulted  in  nine  correlation 
matrices,  one  for  each  combination  of  grade/age  and  subject  area.     Each  of 
these  matrices  were  factored  using  principal  components  with  unities  on  the 
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diagonal.     The  latent  roots  were  examined  and  a  decision  of  the  number  of 
factors  to  rotate  was  made.     For  each  of  the  nine  grade/age^by-subj ect-area 
cases,  the  selected  number  of  factors  were  rotated  orthogonally  to  a  varimax 
solution,  the  results  were  examined  and  questions  were  assigned  to  unique 
factors. 

The  questions  assigned  to  a  factor  were  then  scaled  so  that  the 
responses  to  each  question  were  on  a  1  to  5  scale  and,  if  necessary,  oriented 
by  reversal  so  that  a  score  of  1  corresponded  to  the  most  negative  response 
and  a  score  of  5  to  the  most  positive.     Finally,   the  WARM  composite 
corresponding  to  the  factor  was  defined  as  the  simple  average  of  these  scaled 
and  oriented  responses  to  the  questions  assigned  to  that  factor. 

The  final  step  in  the  creation  of  the  composite  background  variables  was 
the  creation  of  sets  of  student  level  plausible  values  for  each  composite. 
If  a  respondent  had  answered  all  questions  going  into  the  composite,  then 
that  respondent's  WARM  score  would  be  directly  calculable,  without  error,  by 

e  «  w'x 

where  x  is  the  vector  of  the  subject's  responses  to  the  n  questions  in  the 
composite  and  w  is  a  vector  of  n  constants,  each  equal  to  1/n.  However, 
since  each  respondent  is  typically  presented  only  a  subset  of  the  questions, 
a  respondent's  composite  value  must  be  estimated  by  an  application  of  the 
WARM  technology.     Briefly,   the  WARM  technology,  which  is  a  kind  of  multiple 
regression,  produces  for  each  student  a  set  of  plausible  values,  each  of 
which  predicts  what  that  student's  composite  score  might  plausibly  be,  based 
on  the  student's  responses  to  the  questions  in  the  composite  that  were 
presented  to  the  student  and  based  on  the  student's  status  on  the 
conditioning  variables  listed  in  Table  8.4  at  the  end  of  this  chapter. 

Let  x°    represent  the  responses  of  the  i^^  student  to  the  questions  in 
the  composite  which  were  presented  to  that  student  and  let  y^.  be  the  values 
of  that  student's  conditioning  variables.     Then  the  k      plausible  value  of 
the  WARM  composite  i9 ,  based  on  the  student's  observed  responses  and 
conditioning  variables  is 


where 


^ik  ^      plausible  value  of  the  WARM  composite, 

A 

r        is  the  vector  of  estimated  effects  for  the  conditioning 
variables , 

A 

P        is  the  vector  estimated  .^s  giving  the  change  in  the 

composite  variable  for  a  unit  change  in  the  scores  on 
each  of  the  questions  in  x°. ,  with  the  linear  effect  of 
the  conditioning  variables  fteld  fixed. 
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f7^»  ot^]     is  a  random  draw  from  a  N(0,S)  distribution,  where  S 
is  the  estimated  variance -covariance  matrix  of  the 

A  A 

estimates  of  T  and  0  and  reflects  the  uncertainty  due  to 
using  sample  estimates  of  the  regression  equation;  and 

2 

is  an  estimated  residual  drawn  from  a  N(0,a  ) 

distribution  where  o    is  the  variance  of  the  predictive 

distribution  of  the  ^ARM  value  eiven  the  observed  values 

Co 
of  y  .  and  x  . . 


The  parameters  relating  the  responses  on  a  given  set  of  background 

A  A, 

questions  (fi)  and  values  of  the  conditioning  variables  (r)  with  the  means  of 
the  responses  each  of  the  questions  in  the  WARM  composit-  were  estimated  by 
least-squares  technology.     To  accomplish  this  it  is  sufficient  to  obtain 
estimates  of  the  means,  variances,  and  interitem  covariances,  by  conditioning 
subgroup,  for  the  complete  set  of  background  questions  going  into  the 
composite.     Because  the  WARM  composite  is  the  mean  of  the  individual 
questions,  this  in  turn  produces  estimates,  by  conditioning  subgroup,  of  the 
WARM  value  mean  and  variance,  as  well  as  of  the  covariances  between  the  WARM 
composite  and  each  of  the  individual  background  questions.     These  provide  a 
complete  set  of  sufficient  statistics  (the  normal  equations)  for  the  standard 
least-squares  prediction  of  a  WARM  composite  value  given  conditioning 
variable  characteristics  and  responses  to  any  subset  of  the  background 
questions . 

Solving  these  normal  equations  produces  the  standard  least-squares  point 
estimate  of  a  student's  score  on  the  composite,  which  is,   in  the  above 
notation, 

A  A 

?.  «    r  y^.  +  ^  x°.  . 
1  1  1 

This  standard  estimate  does  not  take  into  account  the  distribution  of 
potential  scores  for  any  individual.     In  fact,   9,  is  an  estimate  of  the  mean 
of  the  predictive  distribution  of  potential  ^s  for  the  individual  and,  as 
such,  does  not  address  the  likelihood  of  other  values  from  this  distribution, 
any  one  of  which  might  also  have  been  the  student's  WARM  composite  score  had 
the  student  answered  all  the  questions.     (Note:   for  convenience  we  are 
treating  the  WARM  composite  as  a  continuous  variable;   it  is  in  fact  discrete, 
but  can  take  a  large  number  of  closely  spaced  values.) 

A  check  on  the  impact  of  the  approximations  and  simplifying  assumptions 
employed  in  the  WARM  approach  was  carried  out  with  the  writing  data  from  the 
1984  NAEP  writing  assessment  (Beaton  &  Johnson,  1987)  ,     As  a  comparison  for 
subgroup  average  writing  scores,  the  same  statistics  were  calculated  using  a 
totally  different  approach- -the  model -free,  unbiased  estimate  for  average 
responses  based  on  the  methodology  employed  by  the  Education  Commission  of 
the  States  in  previous  NAEP  analyses.     The  latter  method  is  prohibitively 
expensive  to  be  used  for  all  NAEP  statistics,  but  could  be  calculated  for  the 
44  questions  in  the  common  background  questionnaire.     Beaton  and  Johnson 
found  that  statistics  based  on  the  WARM  were  nearly  indistinguishable  from 
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the  model- free  averages  for  those  subgroups  distinguished  as  conditioning 
variables,  and  for  subgroups  whose  memberships  were  well-predicted  by 
conditioning  variables.     Estimated  standard  errors  were  also  smaller  for  the 
WARM  estimates.     For  those  subgroups  that  were  neither  conditioned  on  nor 
well-predicted  by  conditioning  variables,  the  WARM  exhibited  biases.  The 
nature  of  such  biases  in  plausible  values  methodology  is  discussed  further  ii 
section  8.4.3  of  this  report.     Their  causes,  properties,  and  remedies  are 
discussed  at  length  in  Mislevy  (1988). 


8 . 4  ANALYSES 


When  survey  variables  are  observed  without  error  from  every  respondent, 
standard  variance  estimators  quantify  the  uncertainty  associated  with  sample 
statistics  from  the  only  source,  namely  the  sampling  of  respondents.  Item 
percents -correct  for  NAEP  cognitive  exercises  meet  this  requirement,  but 
scale-score  proficiency  values  do  not.     The  IRT  and  WARM  models  used  in  their 
construction  posit  an  unobservable  proficiency  variable  6  to  summarize 
performance  on  the  items  in  the  area.     The  fact  that  6  values  are  not 
observed  even  for  the  respondents  in  the  sample  requires  additional 
statistical  machinery  to  draw  inferences  about  6  distributions  and  to 
quantify  the  uncertainty  associated  with  those  inferences.     As  described 
above,  we  have  adapted  Rubin's  (1987)  multiple  imputations  procedures  to  the 
context  of  latent  variable  models  to  produce  the  plausible  values  upon  which 
many  analyses  of  the  1986  NAEP  data  are  based.     This  section  describes  how 
plausible  values  were  employed  in  subsequent  analyses  to  yield  inferences 
about  population  and  subpopulation  distributions  of  proficiencies. 


8.4.1    Computational  Procedures 

Even  though  we  do  not  observe  the  6  value  of  respondent  i ,  we  do  observe 
variables  that  are  related  to  it:     x      the  respondent's  answers  to  the 
cognitive  items  he  or  she  was  adjninistered  in  the  area  of  interest,  and  v., 
the  respondent's  answers  to  demographic  and  background  variables.     Suppose 'we 
wish  to  draw  inferences  about  a  number  T(e,Y)  that  could  be  calculated 
explicitly  if  the  6  and  y  values  of  each  member  of  the  population  were  known. 
Suppose  further  that  if  $  values  were  observable,  we  would  be  able  to 
estimate  T  from  a  sample  of  N  pairs  of  6  and  y  values  by  the  statistic  t(e  y) 
[where  (i,^)  -  (^^ .Y^ . • • • . ^^,y^) ] ,  and  that  we  could  estimate  the  variance  in 
t  around  T  due  to  sampling  respondents  by  the  function  U(i,x).     Given  that 
observations  consist  of  (x.,y  )  rather  than  («.,y.),  we  can  approximate  t  by 
its  expected  value  conditional  on  (x,^) ,  or 

t*(x,ir)  -  E[t(i.x)|x.2] 

-  S  t(i,x)  pa|x,x)  di  . 

It  is  possible  to  approximate  t    with  random  draws  from  the  conditional 
distributions  p(^^|x^,y^),  which  are  obtained  for  all  respondents  by  the 
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method  described  above  in  section  8.2.     Let  d    be  the  m      such  vector  of 
"plausible  values,"  consisting  of  a  (possibly  multidimensional)  value  for  the 
latent  variable  of  each  respondent.     This  vector  is  a  plausible 
representation  of  what  the  true  i.  vector  might  have  been,  had  we  been  able  to 
observe  it.     The  following  steps  describe  how  an  estimate  of  a  scalar 
statistic  t(±,20  sampling  variance  can  be  obtained  from  M  (>1)  such 

sets  of  plausible  values.     (Note:     Five  sets  of  plausible  values  were  used  in 
NAEP  analyses  in  each  subject  area,  and  are  provided  on  the  NAEP  public -use 
data  tapes  for  secondary  analysis.) 


1.  Using  each  set  of  plausible  values  $     in  turn,  evaluate  t 
as  if  the  plausible  values  were  true  values  of  $_.  Denote 

A 

the  results  t  ,  for  m«l,...,M. 
m 

2.  Using  the  multiple  weight  jackknife  approach  (see  section 

A 

14.2.1),  compute  the  estimated  sampling  variance  of  t  , 

denoting  the  result  U  . 

m 

3.  The  final  estimate  of  t  is 

M  A 

1_ 

m 


t*  -    Z  t  •  /  M 
m=l 


Compute  the  average  sampling  variance  over  the  M  sets  of 
plausible  values,  to  approximate  uncertainty  due  to 
sampling  respondents: 


*  M 

U    «    Z  U    /  M 
m=l 


5.  Compute  the  variance  among  the  M  estimates  t  ,  to 
approximate  uncertainty  due  to  not  observing°^S  values 
from  respondents: 

B    -        (t  -  t*)^  /  (M«l)  . 
N         T  m 

m«l 

•k 

6.  The  final  estimate  of  the  variance  of  t    is  the  sum  of 


two  components : 


V  -  U*  +  (I+M""^)  B^ 


Note:  Due  to  the  excessive  computation  that  would  be  required, 
NAEP  analyses  did  not  compute  and  average  jackknife  variances 
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over  all  five  sets  of  plausibly  values,  but  only  on  the  first 
set.     Thus,  in  NAEP  reports,  U    is  approximated  by  , 

8.4.2    Statistical  Tests 

Suppose  that  if  $  values  were  observed  for  sampled  students  the 
1/2  ' 
statistic  (t  -  T)/U       would  follow  a  t-distribution  with  d  degrees  of 

freedom.  Then  the  incomplete -data  statistic  (t*  -  T)/V^^^  is  approximately 
t-distributed,  with  degrees  of  freedom  given  by 


-1  9 

(M-1)  (l^r^^r    -    d  ^ 


where  r^^^  is  the  relative  increase  in  variance  due  to  not  observing  6  values: 


When  B    is  small  relative  to  U  ,  the  reference  distribution  for 
incomplete-data  statistics  differs  little  from  the  reference  distribution  for 
the  corresponding  complete-data  statistics.     This  is  the  case  with  main  NAEP 
reporting  variables.   If  in  addition  d  is  large,  the  normal  approximation  can 
be  used  to  flag  "significant"  results. 

For  k-dimensional  ^,  such  as  the  k  coefficients  in  a  multiple  regression 
analysis,  each        and  U    is  a  covariance  matrix,  and  B     is  an  average  of 
squares  aiid  cross-products  rather  than  simply  an  average  of  squares.     In  this 
case,  the  quantity 

(T-t*)  V"-^  (T-t*)' 

is  approximately  F  distributed,  with  degrees  of  freedom  equal  to  k  and  i/, 
with  u  defined  as  above  but  with  a  matrix  generalization  of  r  : 

=  (I+m'-"")  Trace(Bj^U*'-'-)/k  . 

By  the  same  reasoning  as  used  for  the  normal  approximation  for  scalar  t,  a 
chi-square  distribution  on  k  degrees  of  freedom  often  suffices. 
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8.4.3    Biases  in  Secondary  Analyses 


Statistics  t    that  involve  proficiencies  in  a  scaled  content  area  and 
variables  included  in  the  conditioning  variables  y  ,  are  consistent  estimates 
of  the  corresponding  population  values  T.     Statistics  involving  background 
variables  y  that  were  not  conditioned  on,  or  relationships  among 
proficiencies  from  different  content  areas,  are  subject  to  asjmiptotic  biases 
whose  magnitudes  depend  on  the  type  of  statistic  and  the  strength  of  the 
relationships  of  the  nonconditioned  background  variables  to  the  variables 
that  were  conditioned  on  and  to  the  proficiency  of  interest.     That  is,  the 
large  sample  expectations  of  certain  sample  statistics  need  not  equal  the 
true  population  parameters. 

The  direction  of  the  bias  is  typically  to  underestimate  the  effect  of 
nonconditioned  variables.     For  details  and  derivations,   the  interested  reader 
is  referred  to  Beaton  and  Johnson  (1987),  Mislevy  (1988)^  and  Mislevy  and 
Sheehan  (1987,  section  10.3.5).     For  a  given  statistic  t     involving  one 
content  area  and  one  or  more  nonconditioned  background  variables,  the 
magnitude  of  the  bias  is  related  to  (1)  the  extent  to  which  observed 
responses  x  account  for  the  latent  variable  B,  and  (2)  the  degree  to  which 
the  nonconditioned  background  variables  are  explained  by  conditioning 
background  variables.     The  first  factor- -conceptually  related  to  test 
reliability- -acts  consistently  in  that  greater  reliability  reduces  biases  in 
all  secondary  analyses,     (This  salutary  effect  is  roughly  proportional  to  the 
average  niimber  of  items  in  a  content  area  the  respondents  are  administered.) 
The  second  factor  acts  to  reduce  biases  in  certain  analyses  but  increase  it 
in  others.     In  particular. 


High  shared  variance  between  conditioned  and 
nonconditioned  background  variables  mitigates  biases  in 
analyses  that  involve  only  proficiency  and  nonconditioned 
variables,  such  as  marginal  means  or  regressions. 

High  shared  variance  exacerbates  biases  in  regression 
coefficients  or  conditional  effects  for  nonconditioned 
variables,  when  nonconditioned  and  conditioned  background 
variables  are  analyzed  jointly  as  in  multiple  regression. 


In  the  1984  NAEP  reading  assessment,  the  magnitude  of  shrinkage  for  the 
subgroup  means  of  a  background  variable  that  was  not  conditioned  on  averaged 
about  15  percent.     Biases  in  multiple  regressions  that  included  conditioning 
variables  averaged  about  35  percent.     These  values  may  be  taken  as 
approximate  lower  bounds  for  shrinkage  for  similar  analyses  with  total 
reading,  mathematics,  and  science  proficiencies  in  the  1986  assessment,  as 
the  total  proportion  of  variation  in  proficiency  accounted  for  by  item  and 
conditioned-background  responses  is  very  similar- -values  between  80  and  90 
percent,  as  compared  to  the  80 -percent  values  attained  with  1984  reading  (see 
Table  8.1).     Somewhat  higher  degrees  of  shrinkage  are  possible  in  subscale 
areas,  but  may  be  mitigated  by  the  fact  that  four  times  as  many  background 
variables  were  included  in  the  conditioning  vector  this  year.     Thus  more 
analyses  of  important  variables  are  free  from  such  biases  (i.e.,  those 
variables  that  have  been  conditioned  on) ,  which  unequivocally  improves  the 
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Table  8.1 

Proportions  of  Proficiency  Variance  Accounted  For 


Percent 

Variance 

Accounted  for  bv. 

Background 

Background    Items  civen 

Items 

Grade /Age  SralA 

Onlv 

D  a.  Civ  g  IT  O  unci 

Onlv* 

Reading 

.505 

.836 

.669 

O  A  0 

7/13 

Reading 

.508 

*  \J  A.  ^ 

•  OZl 

.769' 

11/17 

Reading 

.450 

Aft  '^ 

.  798 

3/9 

Science  Composite 

.696 

.  827 

.  *+ 

.714 

Life  Science 

.528 

.726 

.418 

.  604 

Nature  of  Science 

.485 

.667 

.354 

Physical  Science 

.509 

.719 

.604 

7/13 

Science  Composite 

.784 

.873 

410 

.  / 

Life  Science 

.564 

.  745 

A  1  Q 

Chemistry 

.526 

.  670 

.  ^  u  ^ 

/i  7  ft 

Nature  of  Science 

.613 

.750 

.355 

Q  Q  "7 
.  JO  / 

Physics 

.467 

.636 

.318 

.  4o  / 

Earth  &  Space 

.552 

.  728 

.  J7  J 

.591 

11/17 

Science  Composite 

.819 

.  892 

.  790 

Life  Science 

.599 

.771 

.428 

Chemistr}^ 

.659 

.774 

.336 

c  n  o 

.  598 

Nature  of  Science 

.608 

.761 

.621 

Physics 

.591 

.745 

.597 

Earth  &  Space 

.576 

.727 

.  Dob 

"i  /7 

Mathematics  Composite 

.681 

.838 

.493 

7  ^ 
.  /DO 

Measurement 

.543 

.736 

.423 

.  o ±o 

High-level  Applic. 

.528 

.  722 

41  1 

.597 

Number  Skills 

.501 

.741 

.650 

7/13 

Mathematics  Composite 

.741 

.890 

.  576 

ft/,  n 

Measurement 

.601 

.781 

.450 

^70 

High-level  Applic. 

.618 

.795 

.462 

.  O^Z 

Number  Skills 

.511 

.806 

.602 

7 

Geometry 

.598 

.730 

.329 

.  550 

11/17 

Mathematics  Composite 

.810 

.926 

.612 

.892 

Measurement 

.657 

.822 

.480 

.730 

High-level  Applic. 

.661 

.860 

.586 

.807 

Number  Skills 

.509 

.793 

.579 

.737 

Geometry 

.672 

.830 

.484 

.740 

Functions 

.711 

.841 

.450 

.739 

♦analogous  to  reliability  in  classical  test  theory 
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situation;  and  those  that  have  not  been  included  are  more  likely  to  be 
related  to  others  that  have,  which  improves  marginal  analyses  of  thosa 
variables.     The  shrinkages  to  be  expected  in  mathematics  and  science 
subscales,  however,  will  be  larger  than  those  for  areas  as  a  whole  due  to 
their  lower  conditional  reliabilities. 


8.4.4    A  Numerical  Example 

In  order  to  provide  a  feel  for  how  plausible  values  are  used  in 
subsequent  analyses,  this  subsection  gives  some  of  the  steps  in  the 
calculation  of  1986  grade-level  reading  means  and  their  estimation-error 
variances . 

The  weighted  mean  of  the  first  plausible  values  of  the  9,793  grade  3 
students  in  the  sample  is  38.09,  and  the  jackknife  variance  of  these  values 
is  .03.     Were  these  values  true  $  values,  then  38.09  would  be  the  estimate  of 
the  mean  and  .03  would  be  the  estimation- error  variance.     The  weighted  mean 
of  the  second  plausible  values  of  the  same  students,  however,   is  37.95;  the 
third,  fourth,  and  fifth  plausible  values  give  weighted  means  of  37.99, 
38,04,  and  37.96.     Since  all  of  these  figures  are  based  on  precisely  the  same 
sample  of  students,  the  variation  among  them  is  due  to  uncertainty  about  the 
students'   ^s,  having  observed  their  item  responses  and  background  variables. 
Taking  the  jackknife  variance  estimj^te  from  the  first  plausible  value,  .03, 
as  our  estimate  U    of  sampling  variance,  and  the  variance  among  the  five 
weighted  means,    ,0034,  as  our  estimate  B  of  uncertainty  due  to  not  observing 
6,  we  obtain  as  the  final  estimate  V  of  total  error  variance  .03  +  (1+5  ) 
.0034  -  .034. 

* 

With  U    and  B  defined  as  above,  and  with  M=«5,  we  may  obtain  values  for 
Rubin's  (1987)  indices  characterizing  the  properties  of  the  plausible- value- 
based  estimate  of  the  grade  3  mean: 

r,  the  relative  increase  in  variance  due  to  the  latency 
of        is  .1372. 

i/^,   the  degrees  of  freedom  associated  with  the  precision 
loss  due  to  the  latency  of  9  if  the  degrees  of  freedom 
for  the  complete-data  statistic  v^ere  infinite,   is  274. 

Corresponding  values  were  also  calculated  for  grade  7  and  grade  11.  The 
results  are  shown  in  Table  8.2. 
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Table  8.2 

Estimation  Error  Variances  and  Related  Coefficients 
for  the  1986  Grade -level  Reading  Assessments 


Grade              U  B 

3  .03  .003 

7  .02  .001 

11  .04  .002 


V  r  1/ 

.034  .137  274 

.022  .088  640 

.043  .071  901 


8.5    OVERVIEW  OF  SCALES  IN  THE  1986  NAEP  ASSESSMENT 

Scale -score  analyses  based  on  IRT  were  carried  out  in  the  following 
subject  areas  in  the  main  sample  (BIB  administration)  of  the  1986  NAEP 
assessment. 

Reading:     1  scale,  scalar  plausible  values. 

Mathematics:     5  subscales,  multivariate  plausible  values. 

Science:     6  subscales,  multivariate  plausible  values. 

-        History:     1  scale,  point  estimates  for  each  student. 

Literature:     1  scale,  point  estimates  for  each  student. 

The  conditioning  variables  employed  in  the  construction  of  plausible  values 
for  reading,  mathematics,  and  science  are  listed  in  Table  8.4.     Table  8.5 
gives  details  of  exactly  how  the  background  effects  were  coded  in  order  to 
produce  the  conditioning  vector  y  .     Conditional  effect  parameters  T  and  the 
associated  residual  covariances  S  were  estimated  separately  in  each  subject 
area  and  in  each  grade/age.     Additional  information  on  these  analyses  is 
presented  in  Chapters  9,  10,  11,  and  13. 

IRT  scale-score  analyses  were  also  carried  out  separately  in  bridge 
samples,  which  differed  substantially  in  administration  procedures  and  are 
not  to  be  merged  with  the  main-sample  scale-score  data: 

Mathematics,  bridge  sample,  paced  administration:     1  scale, 
plausible  values. 

Science,  bridge  sample,  paced  administration:     1  scale,  plausible 
values , 

Additional  information  on  these  analyses  appears  in  Chapters  10  and  11, 
following  the  discussion  of  the  main-sample  analyses. 
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The  weighted  average  response  method  (WAjRM)  was  used  to  construct  sets 
of  composite  background  variables  for  each  grade/age  and  for  each  of  the 
subject  areas  of  reading,  mathematics,  and  science.     Between  4  and  8  WARM 
composites  were  specified  for  each  grade/age  and  subject  area,  each  of  these 
composites  being  defined  as  averages  of  subsets  of  the  background  and 
attitude  questions  related  to  that  subject  area  which  were  also  presented  to 
the  students  of  that  grade/age.     The  number  of  such  composites,  by  grade/age 
and  subject  area,  are  shown  in  Table  8.3.     The  names  of  the  WARM  variables, 
along  with  short  descriptors,  are  given  in  the  subject  area  chapters. 


Table  8.3 
Number  of  WARM  Composite  Variables 
by  Subject  Area  and  Grade /Age 

Grade/Age  Reading         Mathematics  Science 

Grade  3/Age  9  5  5  5 

Grade  7/Age  13  4  6  7 

Grade  11/Age  17  4  6  8 


The  questions  comprising  each  of  the  final  set  of  50  WARM  composites 
appear  in  Appendix  C  in  Tables  C.l,  C.2,  and  C.3  (for  reading);  Tables  0,5, 
C.6,  and  C.7  (for  mathematics);  and  Tables  C.9,  C.IO,  and  C.ll  (for  science). 
The  conditioning  background  variables  used  in  their  construction  appears  in 
Table  8.4.     The  mapping  of  the  original  responses  to  the  questions  to  the 
scaled  and  oriented  responses  used  for  the  WARM  composites  appears  in  Tables 
C.4,  C.8,  and  C.12  respectively  for  reading,  mathematics,  and  science. 
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Table  8.4 


Variables  Conditioned  on  for  1986  by  Grade/Age 


Grade  3 /Age  9 


Overall 

Gender 

Ethnicity 

STOC 

Region 

Parents '  Education 
Items  in  the  Home 
TV  Watching 
Homework 

Home  Language  Minority 

(self)  crossed  with 

Ethnicity 
Percent  in  Lunch  Program 
Percent  White  in  School 
Ethnicity  by  Gender 
Ethnicity  by  Parents' 

Education 
Age  by  Grade 
Public  V.  Private  School 
Family  Asks  About 

Schoolwork 
Went  to  Preschool 
Single/Multiple  Parent  Home 
Mother  at  Home 
Mother  Works  Outside  Home 
Time  Spent  Studying  Science 
Use  Computers  for  Math 
Adult  Supervision  of 

Student  after  School 
First  Quantile* 
Second  Quantile* 
Sample  Type  (Reading  only) 


Grade  7/Ap;e  1!^ 

Overall 

Gender 

Ethnicity 

STOC 

Region 

Parents '  Education 
Items  in  the  Home 
TV  Watching 
Homework 

Home  Language  Minority 
(self)  crossed  with 
Ethnicity 

Percent  in  Lunch  Program 

Percent  White  in  School 

Ethnicity  by  Gender 

Ethnicity  by  Parents' 
Education 

Age  by  Grade 

Public  V.  Private  School 
Family  Asks  About 

Schoolwork 
Went  to  Preschool 
Single/Multiple  Parent  Home 
Mother  at  Home 
Mother  Works  Outside  Home 
Use  Computers  for  Math 
Type  of  Math  Class  In 
Studying  in  Science  this  Year 
Grades  in  School 
First  Quant ile* 
Second  Quantile* 
Sample  Type  (Reading  only) 


Grade  11 /Age  17 

Overall 

Gender 

Ethnicity 

STOC 

Region 

Parents '  Education 
Items  in  the  Home 
TV  Watching 
Homework 

Home  Language  Minority 

(self)  crossed  with 

Ethnicity 
Percent  in  Lunch  Progrsm 
Percent  White  in  School 
Ethnicity  by  Gender 
Ethnicity  by  Parents' 

Education 
Age  by  Grade 
Public  V.  Private  School 
Family  Asks  About 

Schoolwork 
Went  to  Preschool 
Single/Multiple  Parent  Home 
Mother  at  Home 
Mother  Works  Outside  Home 
Grades  in  School 
High  School  Program 
Number  of  Math  Courses 
Number  of  Science  Courses 
Post-Secondary  Plans 
Hours  of  Outside  Work 
Type  of  English  Class  In 
First  Quantile* 
Second  Quantile* 


*  When  reading  is  the  subject  area  being  analyzed,  the  variables  "First 
Quantile*^  and    "Second  Quantile"  refer  to  a  respondent's  booklet-based 
percent -correct  scores  in  mathematics  and  science.     When  mathematics  is 
being  analyzed,  the  variables  refer  to  reading  and  science.     When  science 
is  being  analyzed,  the  variables  refer  to  reading  and  mathematics. 
Quantiles  are  obtained  by  trichotomizing  percentiles;  a  student  is 
designated  as  either  in  the  lowest  quarter,  the  middle  half,  or  the  upper 
quarter.     Students  not  receiving  any  items  of  the  type  in  question  are 
coded  in  the  same  manner  as  those  who  had  received  some  items  and  were 
found  to  be  in  the  middle  half. 
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Table  8.5 


Contrast  Codings  for  1986  Conditioning  Variables 


Contrast 

Variable  Name  Ages  Variable  Coding  Coding^ 

Overall  All  ---  1 

Gender  All  1  Male  0 

2  Female  1 

Ethnicity  All  1  White  000 

2  Black  100 

3  Hispanic  010 

4  Asian  American  001 

5  American  Indian  000 

6  Unclassified  000 
BLK  Missing  000 

STOC  All  1  Low  Metro  00 

2  High  Metro  10 

3  All  Others  and  Missing  01 

Region  All  1  Northeast  000 

2  Southeast  100 

3  Central  010 

4  West  001 

Parents'  Education  All  1  <  High  School  0000 

2  High  School  Grad  1000 

3  Post-High  School  0100 

4  College  Grad  0010 
BLK  Missing  and  1  Don't  Know  0001 

Items  in  the  Hod.c  All  1  0  to  3  of  the  five  items  00 

2  Four  of  the  five  items  10 

(Items  asked  about  are  3  Five  of  the  five  items  01 

Newspaper,  Dictionary,  BLK  Missing  00 

>25  Books,  Encyclopedia, 
and  Magazines.  Three  or 
more  missing  -  Missing.) 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts.     "Items  in  the 
home",  for  example,  induces  two  contrasts:  A  response  of  2  vs.  all  other  responses, 
and  a  response  of  3  vs.  all  other  responses.     Barred  columns  treated  as  one  contrast 
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Table  8.5 
(continued) 


Variable  Name 
TV  Watching 


Ages 


All 


Home  Language  Minority 
by  Ethnicity 


Homework 


All 


13,  17 


Variable  Coding 


1  None 

2  One  hour  or  less  per  day 

3  Two  hours 

4  Three  hours 

5  Four  hours 

6  Five  hours 

7  Six  or  more  hours  per  day 
BLK  Missing 

Yes  and  Hispanic 
Yes  and  Asian  American 
Yes  and  other  Ethnicity 
No  and  Missing 

1  None 

2  <  15  minutes 

3  1/2  hour 

4  One  hour 

5  >  One  hour 
BLK  Missing 

1  Don ' t  have  any 

2  Don' t  do  any 

3  1/2  hour 

4  One  hour 

5  Two  hours 

6  >  Two  hours 
BLK  Missing 


Contrast 
Coding* 


0 
1 
2 
3 
4 
5 
6 
3 


00 
01 
04 
09 
16 
25 
36 
09 


100 
110 
101 
000 

10 

11 

12 
13 
14 
00 

100 
010 
Oil 
012 
013 
014 
000 


Percent  in  Lunch  Program  All 


0 
1 
2 


000  0 

001  0 

002  0 


99 
100 
BLK 


099  0 

100  0 
000  1 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts, 
treated  as  one  contrast. 


Barred  columns 
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Table  8.5 
(continued) 


Contrast 


Variable  Name                       Ages                     Variable  Coding  Coding-^ 

Percent  White  in  School         All               0  -  49  White  Minority  100 

50  -  79  Integrated  010 

80  -  100  Predominantly  White  001 

BIX  Missing  000 

Ethnicity  by  Gender                All               White,  Other,  &  Missing  Male  000 

Black  Male  000 

Hispanic  Male  000 

Asian  American  Male  000 

White,  Other,  &  Missing  Female  000 

Black  Female  100 

Hispanic  Female  010 

Asian  American  Female  001 


Ethnicity  by  Parents' 
Education 


All 


Age  by  Grade 


All 


Missing  <  HS 
Missing  HS  Grad 
Missing  Post  HS 
Missing  Co  Grad 
Missing  Unknown 


White,  Other,  & 
White,  Other,  & 
White,  Other,  & 
White,  Other,  & 
White,  other,  & 
Black  <  HS 
Black  HS  Grad 
Black  Post  HS 
Black  College  Grad 
Black  Unknown 
Hispanic  <  HS 
Hispanic  HS  Grad 
Hispanic  Post  HS 
Hispanic  College  Grad 
Hispanic  Unknown 
Asian  American  <  HS 
Asian  American  HS  Grad 
Asian  American  Post  HS 
Asian  American  College  Grad 
Asian  American  Unknown 

1  <  Modal  Age,  Modal  Grade 

2  Modal  Age,  <  Modal  Grade 

3  Modal  Age,  Modal  Grade;  and 

Missing 

4  Modal  Age,  >  Modal  Grade 

5  >  Modal  Age,  Modal  Grade 


000000000000 
000000000000 
000000000000 
000000000000 
000000000000 
000000000000 
100000000000 
010000000000 
001000000000 
000100000000 
000000000000 
000010000000 
000001000000 
000000100000 
000000010000 
000000000000 
000000001000 
000000000100 
000000000010 
000000000001 

0000 
1000 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts, 
treated  as  one  contrast. 


0100 
0010 
0001 

Barred  columns 
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Variable  Name  Apes 
Public  V.  Private  Schools  All 


Family  Asks  About  School-  All 
work 


Went  to  Preschool 


All 


Single/Multiple  Parent 
Home 


Mother  at  Home 


All 


13,  17 


Mother  Works  Outside  of 
Home 


13,  17 


Table  8,5 
(continued) 


Variable  Coding 


*  Multicoluran  entries  without  overbars  indicate  multiple  contrasts, 
treated  as  one  contrast. 


Contrast 
Coding* 


1  Public 

U 

1 

3  Catholic 

1 

4  Bureau  of  Indian  Affairs 

1 

-/  jL/c^  L.  •   oi.  jL^enense 

1 

BLK  Miqqinp 

1 

1  Almost  Every  Day 

1 

2  Once  a  Week 

0 

3  Oncp  A  Mnnt"h 

u 

4  Never 

0 

BLK  Missine 

u 

1  Yes 

1 

2  No 

0 

3    T    Dor. ' t  ]^nnTiT 

U 

A 

U 

1  Yes  to  Father  and  Mother 

1 

at  Home 

2  Any  Other  Responses 

0 

BLK  Missing 

yj 

1  Works  Outside 

1 

2  Doesn't  Work  Outside 

1 

3  Mother  Not  at  Home 

A 

V 

BLK  Missing 

n 

u 

1  Works  Outside  Full-Time 

1 

2  Works  Outside  Part-Time 

1 

3  Doesn't  Worlc  Ont"€:1Hp 

J. 

4  Mother  Not  at  Home 

n 

u 

BLK  Missing 

n 

1  Works  Outside 

1 

2  Doesn't  Work  Outside 

0 

3  Mother  Not  it  Home 

0 

BLK  Missing 

0 

1  Works  Outside  Full-Time 

1 

2  Works  Outside  Part-Time 

1 

3  Doesn't  Work  Outside 

0 

4  Mother  Not  at  Home 

0 

BLK  Missing 

0 

Barred  columns 
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Table  8.5 
(continued) 


Contrast 

Variable  Name  Ages  Variable  Coding  Coding* 

Time  Spent  Studying  9  1  Daily  10 

Science  2  Several  Times  a  Week  10 

3  Once  a  Week  10 

4  <  Once  a  Week  01 

5  Never  01 
BLK  Missing  00 

Use  Computers  for  Math  9,  13  1  Yes  1 

2  No  0 

BLK  Missl.ng  0 

Adult  Supervision  of  9  1  Yes  1 

Student  after  School  2  No  0 

BLK  Missing  0 

Type  of  Math  Class  In  13  1  None  000 

2  Regular  Math  100 

3  Pre -Algebra  010 

4  Algebra  001 

5  Other  001 
BLK  Missing  000 

Studying  in  Science  this        13  1  None  0000 

Year  2  Life  Science  1000 

3  Physical  Science  0100 

4  Earth  Science  0010 

5  General  Science  0001 

6  Other  0001 
BLK  Missing  0000 

Grades  in  School  13,  17         1  A  4.0 

2  A-B  3.5 

3  B  3.0 

4  B-C  2.5 

5  G  2.0 

6  C-D  1.5 

7  D  1.0 

8  <  D  O.b 
BLK  Missing  2.0 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts.     Barred  columns 
treated  as  one  contrast. 
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Variable  Name 
High  School  Program 


Ages 
17 


17 


17 


17 


Hours  of  Outside  Work 


17 


Type  of  English  Class  In  17 


Table  8.5 
(continued) 

Variable  Coding 

1  General 

2  College  Preparatory 

3  Vocational,  Technical 
BLK  Missing 

0  None  of  the  seven  classes 

1  One  of  the  seven  classes 

2  Two  of  the  seven  classes 

3  Thj^i^^of  the  seven  classes 

4  Four  of  the  seven  classes 

5  Five  of  the  seven  classes 

6  Six  of  the  seven  classes 

7  Seven  of  the  seven  classes 
BLK  Missing 

0  None  of  the  four  classes 

1  One  of  the  four  classes 

2  Two  of  the  four  classes 

3  Three  of  the  four  classes 

4  Four  of  the  four  classes 
BLK  Missing 

1  Work  Full  Time 

2  Two-year  College 

3  Four-year  College 

4  Other 
BLK  Missing 

1  None 

2  <  6  Hours 

3  6  to  10  Hours 

4  11  to  15  Hours 

5  16  to  20  Hours 

6  21  to  25  Hours 

7  26  to  30  Hours 

8  >  30  Hours 
BLK  Missing 

1  Not  Taking 

2  Advanced  Placement 

3  College  Preparatory 

4  General 

5  Remedial 
BLK  Missing 

*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts, 
treated  as  one  contrast. 


Number  of  Math  Courses 
Taken 

(Classes  asked  about: 
General  Business,  Pre- 
Algebra,  1st  year  Al- 
gebra, 2nd  year  Algebra, 
Geometry,  Trigonometry, 
Pre-Calculus,  Calculus.) 


Number  of  Science  Courses 
Taken 

(Classes  asked  about: 
General  Science, 
Biology,  Chemistry, 
Physics . ) 

Post -Secondary  Plans 


Contrast 
Coding* 

00 
10 
01 
00 

0 
1 
2 
3 
4 
5 
6 
7 
0 

0 
1 
2 
3 
4 
0 

00 
10 
01 
00 
00 


00 
10 
10 
00 
01 
00 

Barred  columns 
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Table  8.5 
(continued) 


Contrast 


Variable  Name                        Ages                      Variable  Coding  Coding* 

Reading  Quant ile                     All               0  Missing  0 

1  Lower  Quarter  -1 

2  Middle  Half  0 

3  Upper  Quarter  1 

Math  Quantile                          All               0  Missing  0 

1  Lower  Quarter  -1 

2  Middle  Half  0 

3  Upper  Quarter  1 

Science  Quantile                     All               0  Missing  0 

1  Lower  Quarter  -1 

2  Middle  Half  0 

3  Upper  Quarter  1 

Sample  Type  (Reading  only)     9,  13           1  BIB  1 

2  Bridge  0 


*  Multicolumn  entries  without  overbars  indicate  multiple  contrasts.     Barred  columns 
treated  as  one  contrast. 


FRir 
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CHAPTER  9 
Reading  Data  Analysis 


Chapter  9 
READING  DATA  ANALYSIS'^ 


Rebecca  Zwick 
Educational  Testing  Service 


9.1     SAMPLING  OF  STUDENTS  AND  ITEMS 


In  1986,  reading  items  were  administered  to  subsets  of  the  spiral 
samples  at  all  three  grade/age  levels  and  to  all  students  in  the  Bridge  A 
samples  at  the  two  lower  age  levels.     Further  detail  on  these  samples  is 
given  in  Chapter  1.     In  brief,  the  purpose  of  the  Bridge  A  samples  was  to 
allow  measurement  of  the  effects  of  changes  in  age  definition  and  time  of 
testing.     Since  there  were  no  such  changes  at  grade  11/age  17,  no  Bridge  A 
sample  was  needed.     The  table  below  gives  the  number  of  students  in  each  of 
the  five  samples  listed  above  who  received  at  least  one  reading  block,  the 
total  number  of  reading  scale  items  administered  to  each  sample,  and  the 
number  of  reading  scale  items  common  to  the  1984  assessment. 


Table  9.1 

Reading  Scale  Item  Information,  by  Sample 


No.  of  Reading 
No.  of  Reading  Scale  Items 

Sample  No.  of  Cases  Scale  Items  Common  to  1984 


Grade  3/Age  9  Spiral  11,575  68  48 

Grade  3/Age  9  Bridge  A  6,932  31  31 

Grade  7/Age  13  Spiral  11,171  61  43 

Grade  7/Age  13  Bridge  A  6,200  25  25 

Grade  11/Age  17  Spiral  20,535  61  43 


In  total,  107  reading  scale  items  were  administered  in  1986;  76  of  these 
items  had  also  been  administered  in  1984.     Four  of  these  items  were  open- 
ended;  the  remainder  were  multiple-choice.     Two  of  the  four  were  administered 
at  all  three  age  levels,   the  remaining  two  at  the  two  upper  age  levels  only. 
These  items  were  rated  on  four-  or  five-point  scales  by  professional  judges, 
as  described  in  Chapter  6.2.     For  purposes  of  item  response  theory  (IRT) 
scaling,  tliese  ordinal  scores  were  then  dichotomized  using  rules  provided  by 
reading  experts  (Table  9.2). 


Laurel  Barnett,  David  Freund,  Bruce  Kaplan,  Laura  McCamley,  and 
Minhwei  Wang  provided  statistical  programming.     Robert  Mislevy,  Kathleen 
Sheehan,  and  Eugene  Johnson  provided  consultation  on  IRT  and  WARM  scaling. 
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Table  9.2 

Dichotomization  Rules  for  Open-ended  Reading  Items  Used  in  Scaling 


NAEP  ID 


Cohort 


Score  Range  for 
Valid  Responses 


Scores 
Considered 
Correct 


N003104  Goods  to  Market 
N021301  Jacob 
N021801  Eggplant  I 
N021805  Eggplant  II 


3/9,  7/13,  11/17 
3/9,  7/13,  11/17 
7/13",  11/17 
7/13,  11/17 


1 
1 
1 
1 


5 
4 
5 
4 


3 
2 
3 
3 


5 
4 
5 
4 


Twelve  reading  items  that  had  been  excluded  from  the  reading  scale  in 
1984  were  again  excluded  in  1986.     Also,  in  the  case  of  open-ended  items, 
neither  primary  trait  scores  for  Rater  2  nor  secondary  trait  scores  were 
scaled.     Table  9.3  lists  the  reading  item  scores  excluded  from  the  scale,  as 
well  as  the  reasons  for  exclusion. 


9.2  SCALING 

In  1984,  responses  to  reading  items  were  summarized  in  a  single  reading 
scale,  a  decision  that  was  supported  by  dimensionality  analyses  (see  Beaton, 
1987a  and  Zw5ck,  1987a).     The  IRT  procedures  applied  to  the  1986  reading 
scale  items,  more  than  70  percent  of  which  are  common  to  1984,  are 
essentially  the  same  as  those  used  to  scale  the  spiral  data  in  1984.  Because 
these  methods  are  extensively  documented  in  Beaton  (1987a)  and  in  Chapter  8 
of  this  report,  only  a  brief  outline  of  the  scaling  procedures  is  given  here. 

9.2.1    Steps  in  Scaling  the  1986  Reading  Data 

9.2.1.1    Item  Calibration 

For  all  three  grade/ages  combined,  the  BILOG  program  (Mislevy  &  Bock, 
1982)  was  used  to  obtain  item  parameter  estimates  on  a  provisional  scale, 
based  on  the  three-parameter  logistic  model.     Parameters  were  estimated  even 
for  previously  administered  items;     parameter  values  were  not  assumed  equal 
to  their  1984  values  at  this  phase. 

To  reduce  costs,  a  random  sample  of  students  was  used  for  this  item 
calibration  phase.     Table  9.4  gives  the  number  of  students  in  each  sample 
that  were  included  in  the  BILOG  run. 
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Table  9.4 

Students  Included  in  BILOG  Run,  by  Sample 


Grade /Age 


Spiral 


Bridge 


Total 


3/9 
7/13 
11/17 


4,116 
3,945 
4,612 


497 
443 


4,613 
4,388 
4,612 


Note  that  2,997  students  from  the  Language  Minority  Probe,  a  NAEP  add-on 
project,  were  also  included  in  this  phase  of  analysis,  but  not  in  subsequent 
phases.     Students  in  each  of  the  eight  samples  (the  five  above,  plus  one 
Language  Minority  sample  at  each  grade/age)  were  treated  as  distinct 
subpopul'ations  in  the  BILOG  run.     That  is,   in  estimating  the  item 
parameters,  the  densities  for  these  eight  groups  were  not  assumed  to  be  the 
same.     A  graphical  analysis  of  residuals  was  conducted  to  determine  whether 
it  was  reasonable  to  asstime  common  item  response  functions  for  these  eight 
groups.     For  each  item  and  each  group,  expected  proportions -correct  (see 
Beaton,  1987a,  p.  302)  for  each  of  approximately  eight  proficiency  levels 
were  obtained.     The  departures  of  these  proportions  from  the  common  estimated 
item  response  function  were  examined.     No  major  or  systematic  departures  were 
found,   indicating  that  a  common  item  response  function  was  appropriate  for 
all  groups. 

9.2.1.2  Rescaling  of  Item  Parameters 

Using  the  Stocking-Lord  (1983)  equating  procedure,   implemented  in  the 
TBLT  program  (Stocking,  1986),  the  item  parameters  were  rescaled  by  deriving 
a  linear  equating  function  based  on  items  common  to  1984.     This  function  was 
used  to  rescale  parameter  estimates  for  items  new  to  1986;  the  1984  parameter 
estimates  were  used  for  old  items.     The  equating  procedure  and  a  study  of  the 
error  are  described  in  Appendix  D.     The  final  parameter  estimates  are  given 
in  Tables  E.l,  E.2,  and  E.3  o^  Appendix  E.     Items  common  to  the  1984  scale 
are  indicated  in  the  table.     The  item  parametc"-*^  w^re  treated  as  known  in  all 
subsequent  calculations.     (The  metric  in  which  e  parameters  are  reported 

is  discussed  in  section  9.4,) 

9.2.1.3  Proficiency  Estimation 

Using  the  iterative  method  described  by  Mislevy  (1985)  and  implemented 
in  the  M-GROUP  program  (Sheehan,  1985),  a  reading  proficiency  ($) 
distribution  was  estimated  for  each  individual.     M-GROUP  was  applied  to  each 
grade/age  separately.     Each  iteration  consists  of  the  following  steps: 

1)      Using  the  current  estimates  of  the  regression 

coefficients,  F,  and  error  variance,  a  ,  associated  with 
the  regression  of  6  on  the  conditioning  variables  y^,  the 
distribution  of  6  is  estimated  for  each  individual  (see 


-210- 


ERLC 


202 


Equation  8.6).    The  distribution  is  assumed  normal  with 
mean  Ty    and  variance  a  .     (In  the  first  iteration, 
starting  values  must  be  assumed  for  T  and  a  .) 

2)  For  each  individual,  standard  Bayesian  calculations  (see 
Equation  8.5)  are  used  to  combine  the  (prior) 
distribution  from  a  with  a  smoothed,  normalized 
approximation  to  the  likelihood  based  on  that 
individual's  observed  item  responses,  yielding  a 
posterior  distribution  for  each  individual. 

3)  Five  "plausible  values"  are  randomly  selected  from  each 
individual's  posterior  distribution. 

4)  Tnese  plausible  values  are  used  to  re-estimate  the 
regression  parameters  T  and  a  . 

5)  Steps  1-4  are  repeated  until  the  changes  in        and  the 
elements  of  T  are  sufficiently  small. 

The  final  set  of  five  plausible  values  drawn  from  each  individual's 
distribution  is  then  used  for  estimating  group  statistics. 

The  background  information,  or  conditioning  variables,  used  in  the 
reading  analysis  are  listed  in  Table  8.1  and  the  estimated  effects  are  given 
in  Tables  B.l,  B.2,  and  B.3  of  Appendix  B.     (The  absence  of  an  estimated 
effect  next  to  a  variable  name  indicates  that  the  variable  was  deleted  in 
oraer  to  eliminate  collinearities . )     Note  that  the  spiral  and  Bridge  A 
samples  were  combined  in  the  M-GROUP  analysis  for  the  two  younger  age  groups 
and  that  an  indicator  variable  for  spiral  versus  Bridge  A  sample  was  included 
as  a  conditioning  variable.     (As  noted  above,  there  was  no  bridge  sample  for 
grade  11/age  17.)  or 


9.3     TREND  ANALYSIS 

Comparisons  of  the  1986  reading  results  for  9,  13,  and  17-year-olds  to 
the  1984  results  suggested  a  large  decrease  in  performance  for  9-  and  17- 
year-olds.     These  changes  were  evident  in  the  item  percents-correct    as  well 
as  the  reading  scale  values.     Because  changes  of  this  magnitude  did  not  seem 
credible,  NAEP  did  not  report  trend  results  in  1986.     NAEP  staff  embarked  on 
a  year- long  investigation  of  the  possible  reasons  for  the  1986  results  The 
NAEP  Technical  Advisory  Committee  (now  the  Design  and  Analysis  Committee) 
statistical  staff  from  the  Office  of  Educational  Research  and  Improvement' 
(OERI),  as  well  as  other  eminent  statisticians  and  educational  researchers 
reviewed  the  results  and  helped  to  guide  NAEP's  investigative  efforts  The 
analyses  that  were  performed  to  explore  the  change  in  reading  results  are 
f^^^^ib^^        The  NAEP  1985-86  Readint.  Anomaly:     A  Technical  Rppnr^  (Beaton, 
1988).     Among  the  issues  discussed  are  sampling,  administration,  data  entry 
scoring,  and  scaling.     Although  the  source  of  the  change  could  not  be 
conclusively  determined,  NAEP  is  now  collecting  data  that  will  provide 
further  information  about  the  puzzling  results  in  1986.     The  1988  data 
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collection  includes,  at  each  age  level,  two  special  supplementary  samples  in 
addition  to  the  1988  sample.     These  additional  samples  duplicate  as  closely 
as  possible  the  1984  and  1986  assessments,  respectively,  in  terms  of  sample 
definition,  item  booklets,  and  dates  and  conditions  of  administration.  By 
comparing  the  reading  results  obtained  from  the  three  samples,  estimates  can 
be  obtained  of  the  effects  of  changes  introduced  in  1986  and  in  1988.  If 
these  effects  prove  to  be  nonnegligible ,  NAEP  plans  to  use  the  data  from 
these  supplementary  samples  as  a  basis  for  adjusting  the  1986  and  1988 
results  so  that  they  can  reported  in  the  same  metric  as  the  1984  scale. 

Although  the  1986  and  1984  assessments  differed  in  some  ways  and  may  not 
be  comparable,  the  1986  findings  resemble  past  assessments  and  other  reading 
measures  in  terms  of  patterns  of  subgroup  difference,  relation  of  reading 
proficiency  to  background  variables,  and  relation  of  reading  proficiency  to 
performance  in  other  subject  areas,  such  as  math  and  science.     It  was 
therefore  decided  to  release  a  cross -sectional  report  only,  using  a  scale 
metric  that  differed  from  the  1984  reading  scale. 

9.4    CROSS -SECTIONAL  ANALYSIS 

The  1986  reading  cross-sectional  report,  Who  Reads  Best?  (Applebee, 
Langer,  and  Mullis ,  1988),   is  based  on  the  students  in  the  spiral  samples  who 
were  in  grades  3,   7,  and  11.     The  reading  scale  was  standardized  to  have  a 
mean  of  50  and  a  standard  deviation  of  10  for  the  three  grade  samples 
combined.     To  obtain  results  in  terms  of  this  metric,   the  scale  values  that 
would  be  obtained  by  using  the  item  parameter  estimates  in  Table  E.l,  E.2, 
and  E.3  of  Appendix  E  need  to  be  transformed.     Letting  6  represent  the 
proficiency  metric  that  corresponds  to  the  item  parameters  provided  and 
letting  RS  represent  the  metric  of  the  reporting  scale,  the  required 
transformation  is  RS  *  10^  +  50.     The  corresponding  changes  to  the  item 
parameters  are  a^^^  ^  a  /lO  and  b^^^  =        Q  ***  '*^RS  *"  ^fi'^  Appendix  D 

for  an  explanation  of  the  linkage  between  the  6  metric  ana  the  1984  reading 
scale . 

Who  Reads  Best?  contains  analyses  relating  reading  proficiency  to 
background  and  attitude  variables  and  to  styles  of  reading  instruction  (as 
reported  by  the  student).     Several  composites  of  background,  attitude,  and 
instruction  variables  were  created,  using  the  WARM  method  described  in 
section  8.3.     The  questions  comprising  each  of  the  reading  WARM  composites 
are  given  in  Tables  C.l,  C.2,  and  C.3  of  Appendix  C.     The  score  values  that 
were  assigned  to  the  original  responses  to  these  questions  for  purposes  of 
deriving  the  WARM  scales  are  given  in  Table  C.4  of  Appendix  C. 
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CHAPTER  10 
Mathematics  Data  Analysis 


Chapter  10 
MATHEMATICS  DATA  ANALYSIS 


Eugene  G.  Johnson 
Educational  Testing  Service 


This  chapter  describes  the  analyses  carried  out  on  the  responses  to  the 
cognitive  and  background  items  in  the  1973-74,  1977-78,  1981-82  and  1986 
assessments  of  mathematics  which  lead  to  the  results  presented  in  The 
Mathematics  Report  Card:     Are  We  Measuring  Up?  (Dossey,  Mullis,  Lindquist,  & 
Chambers,   1988).     The  emphasis  is  on  the  methods  and  results  of  the 
procedures  used  to  develop  the  IRT-based  scale-scores  that  formed  the  basis 
of  that  report.     The  theoretic  underpinnings  of  the  particular  techniques 
discussed  in  this  chapter  are  given  in  Chapter  8. 

The  techniques  required  to  develop  scale-scores  for  the  cross  -  sectional 
analysis  of  the  data  from  the  1986  BIB-spiral  assessment  were  different  from 
the  techniques  required  to  develop  scale -scores  for  the  analysis  of  trends  in 
mathematics  achievement.     Accordingly,  these  two  analyses  are  presented  in 
separate  sections. 

Section  10.1  pertains  to  the  scaling  of  the  data  from  the  spiral 
administration  and  describes  the  creation  of  mathematics  content  area 
subscales  as  well  <^s  a  weighted- average  composite  over  the  subscales.  Also 
discussed  in  this  section  are  the  procedures  for  the  empirical  behavioral 
anchoring  that  was  carried  out  on  the  mathematics  composite  and  the  crec.tion 
of  the  WARM  composites  of  mathematics  background  questions.     The  techniques 
used  to  develop  scale -scores  for  the  measurement  of  trend  in  mathematics 
achievement  are  discussed  in  section  10.2. 


10.1     SCALING  OF  THE  CROSS -SECTIONAL  MATHEMATICS  DATA 

The  data  from  the  spiral  assessment  of  mathematics  in  1986  were  used  for 
cross -sectional  analyses  comparing  the  levels  of  mathematics  achievement  for 
various  subgroups  of  the  1986  target  populations.     It  included  three  student 
cohorts:     students  who  were  either  in  the  3rd  grade  or  9-years-old;  students 
who  were  either  in  the  7th  grade  or  13-years-old;  and  students  who  were 
either  in  the  11th  grade  or  17 -years -old.     So  that  the  modal  grades  for  the 
three  age  groups  would  be  the  3rd,  7th,  and  11th  grades,  the  birthdate  ranges 
for  age-eligible  students  were  established  as  October  1976  to  September  1977 


Data  analysis  and  scaling  were  performed  by  David  Freund,  Maxine 
Kingston,  Edward  Kulick,  Joling  Liang,  Laura  McCamley,  Jennifer  Nelson,  Norma 
Norris,  Alfred  Rogers,  and  Minhwei  Wang.     Robert  Mislevy,  Kathleen  Sheehan, 
and  Kentaro  Yamamoto  provided  consultation  on  IRT  scaling. 


-215- 

ERLC 


203 


for  age  9,  October  1972  to  September  1973  for  age  13,  and  October  1968  to 
September  1969  for  age  17.     The  sampled  students  in  each  of  these  thr^e 
cohorts  were  assessed  in  the  spring.     (See  Chapter  3  for  a  description  of  the 
target  populations  and  the  sample  design  used  for  the  assessment) . 

The  pool  of  items  used  in  the  1986  mathematics  assessment  contained  a 
range  of  open-ended  and  multiple - choice  questions  measuring  performance  on 
sets  of  objectives  developed  by  nationally  representative  panels  of 
mathematics  specialists,  educators,  and  concerned  citizens  and  documented  in 
Math  Objectives,  1985-86  Assessment  (NAEP,  1986a).     The  objectives  defined 
seven  con+:ent  areas  and  five  process  areas.     The  content  areas  were: 

1)  fundamental  methods  of  mathematics; 

2)  discrete  mathematics; 

3)  data  organization  and  interpretation; 

4)  measurement; 

5)  geometry; 

6)  relations,   functions,  and  algebraic  expressions;  and 

7)  numbers  and  operations. 

The  process  areas  were: 

1)  problem  solving/reasoning; 

2)  routine  applications; 

3)  understanding/comprehension; 

4)  skills;  and 

5)  knowledge. 

A  total  of  537  distinct  mathematics  items,  addressing  the  above 
objectives,  was  administered  in  1986  using  a  BIB-spiral  design  (Messick, 
Beaton,  &  Lord,  1983;  Beaton,   1987a)  to  allocate  the  items  to  the  assessed 
students  (see  Chapter  3) .     In  this  design,   the  entire  1986  assessment  battery 
(including  all  subject  areas  assessed)  was  divided  into  blocks  of 
approximately  15  minutes  each,  and  each  student  was  administered  a  booklet 
containing  three  blocks  of  content  area  materials  as  well  as  a  six-minute 
block  of  background  questions  common  to  all  students.     Seven  blocks  of 
mathematics  questions  were  administered  at  grade  3/age  9,  nine  blocks  at 
grade  7/age  13,  and  eleven  blocks  at  grade  11/age  17.     At  grade  3/age  9,  52 
different  booklets  were  prepared.     Thirty- four  of  them  contained  one  or  more 
mathematics  blocks,  with  each  of  the  seven  blocks  appearing  in  six  or  eight 
booklets.     Sixty-eight  booklets  were  assessed  at  grade  7/age  13,   38  of  which 
contained  mathematics  blocks;  each  mathematics  block  appeared  in  six  to  nine 
different  booklets.     Mathematics  items  were  included  in  41  of  the  96  booklets 
administered  to  students  at  grade  11/age  17,  with  each  block  appearing  seven 
to  nine  times . 


10.1.1    Definition  of  Subscales 

The  analysis  of  the  results  of  the  spiral  assessment  was  carried  out 
using  five  subscales,  each  measuring  a  facet  of  mathematics,  along  with  a 
composite  defined  as  a  weighted  average  of  the  subscales.     The  aim  in  the 
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creation  of  subscales  was  to  facilitate  capturing  essential  subdivisions  of 
mathematics,  as  indicated  by  the  mathematics  Learning  Area  Committee,  in 
order  to  allow  the  detection  of  potential  differences  in  performance  patterns 
between  those  subdivisions ♦ 

The  basis  for  the  definition  of  the  subscales  ultimately  created  was  the 
content  areas  and  process  areas  defined  by  the  mathematics  learning  area 
coimnittee.     In  selecting  subscale  definitions,   it  was  necessary  to  balance 
two  requirements.     Our  aim  was  to  scale  subareas  which  were  as  narrowly 
defined  as  possible  so  that  the  assumptions  of  the  scaling  models  are  most 
nearly  met  and  so  that  the  capability  to  identify  important  interactions 
within  subareas  of  mathematics  would  be  maximized.     Countering  this  aim  was 
the  requirement  that  the  number  of  items  taken  by  any  individual  student 
within  a  subarea  be  sufficient  to  support  currently  available  scaling 
procedures.  This  latter  requirement  precluded  the  possibility  of  scaling 
within  each  of  the  35  content  by  process  cells  since,   in  most  of  these  cells, 
the  small  number  of  items  taken  by  any  individual  student  was  insufficient  to 
support  current  scaling  technology. 

As  a  result,  we  selected  the  content  areas  as  the  initial  basis  of  the 
subscale  definition.     The  numbers  of  items,  by  content  area,  are  shown  in 
Tables  10.1,  10.2  and  10.3  for  grade  3/age  9,  grade  7/age  13  and  grade  11/age 
17,  respectively.     (The  splitting  of  the  numbers  and  operations  content  area 
into  two  parts  is  discussed  below) .     Also  included  in  the  tables  is 
information  about  the  distribution  of  the  numbers  of  items  taken  by 
individual  students  within  each  content  area.     Since  each  student  takes  a 
single  booklet,  this  information  is  presented  in  terms  of  the  distribution  of 
items  per  booklet.     Included  in  the  tables  for  each  content  area  are  the 
average  numbers  of  items  per  booklet  and  the  numbers  of  booklets  with  1-2, 
3-5,  and  more  than  5  items  within  the  content  area. 

Based  on  the  counts  shown  in  the  tables,  we  decided  that  it  was  possible 
to  analyze  and  report  the  following  subscales : 


Grade/Age 
3/9  7/13  11/17 


Fundamental  Methods  X  XX 

Measurement  X  XX 

Data  Organization  and  Interpretation  X  XX 

Geometry  X  X 

Relations  and  Functions  X 
Numbers  and  Operations: 

Higher-level  Applications  X  XX 
Numbers  and  Operations: 

Knowledge  and  Skills  X  XX 


Items  in  the  remaining  content  categories  (discrete  mathematics  for  all 
grade/ages,  geometry  for  grade  3/age  9,  and  relations  and  functions  for  grade 
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Distribution  of  the  Members  of  Items  by 
Booklet  Within  Each  of  the  Mathematics  Content  Areas 

Grade  3/Age  9 


Average 
Number      Number  of  Booklets 


of 

with 

iNumuei. 

Total 

of 

per 

1-2 

3-5 

more 

Area 

Items 

Booklets 

Booklet 

Items 

Items 

Items 

Fundamental  Methods 

102 

25 

4.1 

9 

8 

8 

Discrete  Mathematics 

18 

11 

1.6 

10 

1 

0 

Data  Organization  and 

Interpretation 

96 

19 

5.1 

3 

10 

6 

Measurement 

162 

28 

5.8 

9 

6 

13 

Geometry 

36 

11 

3.3 

5 

5 

1 

Relations ,  Functions ,  and 

Algebraic  expressions 

48 

25 

1.9 

20 

5 

0 

Numbers  and  Operations : 

Higher- level  Applications 

156 

28 

5.6 

9 

6 

13 

Numbers  and  Operations : 

Knowledge  and  Skills 

180 

25 

7.2 

9 

0 

16 
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Table  10.2 


Distribution  of  the  Members  of  Items  by 
Booklet  Within  each  of  the  Mathematics  Content  Areas 

Grade  7/Age  13 


Average 


Nximber 

of 

Number 

of  Booklets 

Number 

Items 

with 

more 

than 

lotai 

of 

per 

1-2 

3-5 

6 

Area 

X  terns 

JDOOKXe  L. 

Items 

Itrms 

Items 

Fundamental  Methods 

150 

25 

6.0 

8 

4 

13 

Discrete  Mathematics 

75 

26 

2.9 

13 

10 

3 

Data  Organization  and 

Interpretation 

147 

28 

5.3 

6 

9 

13 

Measurement 

306 

32 

9.6 

4 

7 

21 

Geometry 

168 

30 

5.6 

6 

12 

12 

Relations,  Functions,  and 

Algebraic  Expressions 

91 

30 

3.0 

14 

12 

4 

Nximbers  and  Operations : 

Higher- level  Applications 

455 

32 

14.2 

2 

2 

28 

Nximbers  and  Operations : 

Knowledge  and  Skills 

396 

32 

12.4 

6 

0 

26 

/ 
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Table  10.3 


Distribution  of  the  Members  of  Items  by 
Booklet  Within  each  of  the  Mathematics  Content  Areas 

Grade  11/Age  17 


Average 
Number 

of  Number  of  Booklets 


Numbf^r 

Urn  ^^lllO 

with 

more 

than 

Total 

of 

per 

1-2 

3-5 

6 

Area 

Items 

Booklets 

Booklet 

Items 

Items 

Items 

Fundamental  Methods 

287 

35 

8.2 

4 

9 

22 

Discrete  Mathematics 

139 

35 

4.0 

9 

18 

8 

Data  Organization  and 

Interpretation 

183 

32 

5.7 

3 

14 

15 

Measurement 

355 

42 

8.5 

4 

6 

32 

Geometry 

325 

39 

8.3 

2 

11 

26 

Relations,  Functions,  and 

Algebraic  Expressions 

370 

40 

9.3 

5 

12 

23 

Numbers  and  Operations : 

Higher- level  Applications 

508 

42 

12.1 

1 

8 

33 

Numbers  and  Operations : 

Knowledge  and  Skills 

523 

39 

13.4 

6 

3 

30 
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3/age  9  and  grade  7/age  13)  were  not  included  in  any  subscales,  since 
consultation  with  mathematics  experts  indicated  that  it  would  be 
inappropriate  to  recluster  these  items  into  other  content  categories. 

Because  there  were  so  many  items  in  the  numbers  and  operations  content 
area,  we  felt  that  it  would  be  feasible  and  preferable  to  split  these  items 
into  two  subclassifications  defined  by  process  area.     We  thus  created  the  two 
subclass if icat ions  of  numbers  and  operations: 

1)  Higher-level  applications - -consisting  of  the  problem 
solving/reasoning,  routine  application,  and 
understanding/comprehension  process  areas;  and 

2)  Knowledge  and  skills  - -consisting  of  the  skills  and 
knowledge  process  areas , 

This  resulted  in  seven  subscale  areas  encompassing  a  total  of  446  unique 
noncalculator  items.     (Since  items  requiring  the  use  of  a  calculator  involve 
somewhat  different  skills  than  items  which  do  not  allow  the  use  of  a 
calculator,  and  since  there  were  too  few  items  per  respondent  to  support  a 
subscale,  the  calculator  items  presented  in  the  1986  spiral  assessment  were 
excluded  from  the  scaling  process.     Analyses  of  achievement  involving  these 
calculator  items  were  based  on  the  percent-correct  metric.) 

10.1.2      Estimation  of  Item  Parameters  for  the  Subscales 

The  next  step  in  the  scaling  process  was  the  estimation  of  item 
parameters  for  the  items  in  each  of  the  seven  defined  subscales.     This  item 
calibration  was  performed  separately  for  each  of  the  seven  subscales,  using 
data  from  all  of  the  grade/age  populations  for  which  the  subscale  was 
defined.     Thus,  five  of  the  subscales,  which  were  defined  for  all  three 
grade/age  populations,  were  calibrated  using  data  from  all  three  grade/ages. 
The  geometry  subscale  was  calibrated  using  data  from  the  two  grade/ages  for 
which  it  was  defined,  grade  7/age  13  and  grade  11/age  17.     Since  the 
relations  and  functions  subscale  was  defined  only  for  grade  11/age  17,  only 
those  students  were  used  in  the  estimation  of  its  item  parameters. 

The  calibration  for  each  subscale  used  the  BILOG  program  and  was 
performed  on  an  approximately  quarter  sample  of  all  the  available  subjects, 
resulting  in  approximately  1,000  examinees  in  each  grade/age  for  each  item. 
(See  Beaton,  1987a,   for  further  description  of  the  calibration  process.) 

In  the  course  of  calibration,  item  fit  was  evaluated  by  inspecting 
residuals  from  the  fitted  item  response  curves.     In  this  inspection,  the 
expected  proportions  of  correct  responses  to  the  item  for  each  grade/age  c.t 
various  points  along  the  subscale  were  compared  with  the  fitted  three - 
parameter  logistic  item  response  curve.     The  expected  proportions  were 
calculated  without  assuming  any  functional  form.     As  a  result  of  these 
examinations,  11  items  were  identified  as  displaying  differential 
subpopulati on  functioning,  in  that  the  expected  proportions  for  one  of  the 
grade/age  populations  deviated  significantly  from  the  item  response  curve. 
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Each  of  these  11  items  were  removed  from  the  scaling  of  the  offending 
grade/age  group  but  were  included  in  the  scaling  of  the  other  grade/age 
groups  who  were  presented  the  item  and  whose  expected  proportions  did  not 
appear  aberrant.     These  items,  along  with  the  grade/age  group  excluded, 
appear  in  Table  10.4. 

An  additional  79  items  were  removed  from  the  scaling  process  because  of 
very  high  rates  of  students  not  reaching  the  item.     (An  unanswered  item 
occurring  after  the  last  valid  response  in  a  block  is  considered  not 
reached.)     Table  10.5  shows  the  distribution,  by  subscale,  of  the  item  level 
rates  of  nonresponse.     The  79  items,  which  were  excluded  from  the  scaling  at 
all  grade/ages,  had  a  nonresponse  rate  of  at  least  45  percent  and  are  listed 
in  Table  10.6.     The  cutoff  value  of  45  percent  was  selected  for  utilitarian 
reasons  in  order  to  ensure  that  at  least  10  items  remained  in  each  subscale 
at  each  grade/age  level.     Because  we  were  concerned  about  the  effect  that  the 
exclusion  of  these  parameters  would  have  on  the  ultimate  subscales,  we 
compared  the  estimates  of  the  item  difficulty  parameters  for  the  retained 
items  when  these  high  nonresponse  items  were  included  in  the  calibration  with 
the  equivalent  item  difficulties  estimated  after  excluding  the  high 
nonresponse  items.     Very  little  difference  in  the  item  difficulties  were 
found;  the  two  sets  of  estimates  when  plotted  against  each  other  lay  very 
tightly  along  a  line  of  unit  slope  through  the  origin. 

The  result  at  the  end  of  the  calibration  phase  was  seven  subscales 
consisting  of  a  total  of  367  items,  distributed  by  grade/age  and  subscale  as 
shown  in  Table  10.7.     For  reasons  given  in  the  next  section,   two  of  these 
subscales  (fundamental  methods  and  data  organization  and  interpretation)  were 
later  excluded  from  the  computation  of  plausible  values.  A  list  of  the  items 
scaled  in  the  five  retained  subscales,  along  with  their  item  parameters 
appears  in  Tables  E.4  through  E.15  in  Appendix  E.     (These  parameters  are  in 
the  metrics  of  the  original  calibration  of  the  subscales.     See  section  10.1.5 
for  the  transformation  of  the  item  paramecers  into  the  metric  of  the  final 
reporting  subscales.) 

10.1.3    Generation  of  Multivariate  Plausible  Values  for  the  Subscales 

The  next  step  in  the  scaling  of  mathematics  was  the  generation  of 
plausible  values  for  each  of  the  subscales.     The  construction  of  plausible 
values  for  the  subscales  used  the  multivariate  procedures  given  in  section 
8.3.3.     As  described  there,  since  a  multivariate  distribution  was  assumed  for 
the  proficiencies  of  an  individual  across  the  various  subscales, 
plausible  values  were  generated  as  vectors.     That  is,  the  first  plausible 
values  (one  for  each  subscale)  for  a  given  examinee  were  generated  as  a  draw 
from  the  multivariate  distribution  p(£|x.,y.)»  where  £  is  a  vector,  x.  are 
the  subject's  item  responses  to  all  items  in  all  subscales,  and  y.  are  the 
subject's  observed  values  on  the  conditioning  variables. 
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Table  10.4 

Mathematics  Items  Deleted  for  Specified  Populations 
Because  of  Differential  Functioning 

BILOG         Subscale  Subscale  Age  Group  Age  Group 


NAEP  ID 

Item  No. 

Number 

Name 

Deleted 

Remaining 

N262501 

51 

1 

Fundamental  Methods 

9 

13,  17 

N262502 

52 

1 

Fundfimental  Methods 

9 

13.  17 

N263501 

85 

2 

Data  Organization 

9 

13,  17 

N236401 

92 

3 

Measurement 

13 

9 

N269001 

144 

3 

Measurement 

9 

13,  17 

N208301 

329 

4 

Numbers  and  Operations: 
Higher- level  Applications 

13 

17 

N287101 

405 

5 

Numbers  and  Operations: 
iN.riowj.eQge  fina  oKxj.is 

13 

17 

N287102 

406 

5 

Niimbers  and  Operations : 
Knowledge  and  Skills 

13 

17 

N286501 

443 

5 

Niimbcra  and  Operations: 
Knowledge  and  Skills 

13 

17 

N286502 

444 

5 

Numbers  and  Operations : 
Knowledge  and  Skills 

13 

17 

N219301 

193 

6 

Geometry 

13 

17 
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Table  10.5 


Number  of  Items  by  Subscale  and 
Percent  Not  Reached 
All  Grade/Ages 


Percent  Not  Reached 

45%  and 


Subscale 

0-9% 

10-19% 

20-29% 

30-44% 

Over 

Tot; 

Fundamental  Methods 

14 

6 

11 

5 

16 

52 

Data  Organization  and 

Interpretation 

19 

4 

2 

3 

8 

36 

Measurement 

30 

16 

12 

9 

5 

72 

Geometry 

18 

9 

7 

6 

3 

43 

Relations,  Functions,  and 

Algebraic  Expressions 

20 

8 

4 

6 

9 

47 

Numbers  and  Operations : 

Higher -level  Applications 

35 

14 

20 

9 

27 

105 

Numbers  and  Operations: 

Knowledge  and  Skills 

58 

12 

_4 

_6 

11 

91 

Total 

194 

69 

60 

44 

79 

ERIC 
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Table  10.6 


Mathematics  Items  Deleted  Because  Percent  Nonresponse  >  45 
(Based  on  Calibration  File) 

Fundamental  Methods 


N203701 
N219701 
N227401 
N221201 


N216901 
N220201 
N228301 
N221601 


N217701 
N220301 
N221101 
N221701 


N219501 
N220601 
N220901 
N221801 


Data  Organization  and  Interpretation 


N224401 
N229201 


N263001 
N229202 


N231301 
N229203 


N225001 
N224301 


Measurement 


N230601 
N269401 


' N217801 


N232601 


N219401 


Numbers  and  Operations:     Higher- level  Applications 


N200701 
N206301 
N203001 
N204801 
N204701 
N278902 
N285301 


N200702 
N227201 
N237501 
N205001 
N207701 
N278903 
N237401 


N273901 
N258501 
N204201 
N230301 
N235301 
N278904 
N207501 


N201801 
N256801 
N258901 
N263801 
N278901 
N279401 


Numbers  and  Operations:     Knowledge  and  Skills 


N283001 
N256801 
N278902 


N201201 
N258901 
N278903 


N201301 
N260301 
N278904 


N273901 
N278901 


Geometry 
N253201 


N233101 


N229601 


Relations  and  Functions 


N255801 
N212101 
N212601 


N230001 
N212201 


N211001 
N212301 


N211501 
N212501 
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Table  10.7 


N^imber  of  Mathematics  Items  Scaled 
by  Subscale  and  Grade/Age 


Sub scale 

Total 

\3  JL  due     J / 

Ap-p  9 

\j  JL  due     /  / 

Age  13 

r-roHo  11/ 
Ap^p  1  7 

Fundamental  Methods 

36 

15 

19 

21 

Data  Organization  and 

Tnt"PT"nTPt";?t"ion 

J.  ^ 

1  6 

1 

J.  o 

Measurement 

67 

26 

45 

39 

Geometry 

40 

24 

37 

Relations  and  Functions 

38 

38 

Numbers  and  Operations: 

Higher-level  Applications 

78 

23 

50 

55 

Numbers  and  Operations: 

Knowledge  and  Skills 

80 

30 

56 

56 

Total 

367 

109 

210 

262 

Originally,  all  seven  subscales  were  included  in  the  generation  of  the 
plausible  values.     However,  the  results  of  this  initial  wave  of  estimation 
indicated  that  the  approximations  necessary  to  accomplish  multivariate 
scaling  with  currently  available  resources  and  techniques  produced 
unacceptable  results  in  the  case  of  two  of  the  subscales. 

Table  10.8  shows  the  means  and  standard  deviations,  by  grade/age  group, 
for  the  first  plausible  value  of  each  of  the  seven  subscales.     If  the 
estimation  procedure  is  operating  correctly,   these  reaps  and  standard 
deviations,  which  are  based  on  the  multivariate  plausible  values,   should  be 
close  to  the  grade/age  group  means  and  standard  deviations  obtained  when  each 
subscale  is  fit  separately.     The  univariate  estimates  of  the  subscale  means 
and  standard  deviations  appear  in  the  columns  headed  by  "BILOG"   (since  they 
are  obtained  as  a  byproduct  of  the  univariate  BILOG  item  calibration  runs). 
It  can  be  seen  from  the  table  that  the  correspondence  between  the  two  sets  of 
estimates  is  generally  good  for  the  bottom  five  subscales  listed  in  the  table 
(i.e.  measurement  and  below)  but  that  the  correspondence  is  noticeably  poorer 
for  the  first  two  subscales  (fundamental  methods,  data  organization  and 
interpretation).     For  both  of  these  subscales,  the  estimated  standard 
deviation  based  on  the  plausible  values  is  markedly  larger  within  each 
grade/age  group  than  the  corresponding  univariate  estimate.  Additionally, 
the  differences  between  the  means  of  the  two  estimates  tends  to  be  larger  for 
these  two  subscales  than  for  the  remaining  five  subscales,  particularly  for 
the  grade  11/age  17  students. 
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These  discrepancies  are  likely  due  to  the  fact  that  relatively  few  items 
within  these  two  subscales  are  taken  by  any  individual.     This  could  mean  that 
the  normalized- likelihood/normal  posterior  approximation  which  was  used  in 
the  generation  of  plausible  values  is  not  performing  adequately  for  these  two 
subscales . 

Because  a  multivariate  scale  was  desired,  and  because  time  and  resources 
precluded  the  development  of  improved  approximations  to  the  multivariate 
posterior  distribution  of  the  proficiencies,  the  decision  was  made  to  drop 
the  fundamental  methods  and  the  data  organization  and  interpretation 
subscales  from  the  analysis.     New  sets  of  vectors  of  plausible  values  were 
generated  for  the  five  retained  subscales.     The  estimation  of  plausible 
values  was  conducted  independently  for  each  of  the  three  grade/age  groups, 
estimating  vectors  of  plausible  values  for  all  subscales  defined  for  a  given 
grade/age.     That  is,  three  subscales  are  defined  for  grade  3/age  9,  four 
subscales  for  grade  7/age  13,  and  five  subscales  for  grade  11/age  17,  The 
conditioning  variables  and  the  estimated  conditioning  effects  (the  T  of 
equation  8.6)  are  given  in  Tables  B.4  through  B.9  in  Appendix  B  for  the  three 
grade/age  groups.     (The  values  of  the  conditioning  effects  are  in  the  metrics 
of  the  original  calibration  of  the  subscales.     Section  10.1.5  provides  the 
transformation  to  the  metric  of  the  final  reporting  subscales). 


10.1.4    Initial  Resolution  of  Linear  Indetermlnacles  of  the  Subscales 

Like  all  IRT  scales,  the  mathematics  subscales  have  a  linear 
indeterminacy  which  may  be  resolved  by  an  arbitrary  choice  of  the  origin  and 
unit- size  in  each  given  subscale.     In  the  course  of  initial  estimation  of  the 
item  parameters  and  the  plausible  values,   the  origin  and  unit-size  for  each 
of  the  subscales  was  provisionally  set  so  that  the  proficiencies  for  that 
subscale  would  have  a  mean  of  zero  and  a  standard  deviation  of  one  over  the 
populations  for  which  that  subscale  was  defined.     Because  three  of  the 
subscales  spanned  all  three  grade/age  groups,  while  one  subscale  was  only 
defined  for  grade  7/age  13  and  grade  11/age  17  and  one  subscale  was  defined 
only  for  grade  11/age  17,  this  initial  choice  of  origin  and  unit-size 
resulted  in  different  means  and  standard  deviations  across  the  subscales  for 
the  three  grade/age  groups,  as  shown  in  Table  10.8.     For  the  purposes  of 
comparing  achievement  between  subscales,  as  well  as  to  facilitate  the 
definition  of  a  mathematics  composite,   it  is  useful  to  have  comparable 
origins  and  unit-sizes  for  each  of  the  five  subscales.     This  was  accomplished 
in  two  steps.     In  the  first  step,   intermediate  transformations  of  each  of  the 
subscales  were  applied  so  that  the  age  group  differences  across  the  various 
subscales  would  be  approximately  equal  to  each  other.     In  the  second  step, 
the  subscales  were  additionally  transformed  to  match  the  units  of  the 
composite,  which  is  defined  in  the  next  section. 

For  the  three  subscales  that  spanned  all  three  grade/ages,  the 
intermediate  transformation  was  accomplished  by  matching  the  age  9  and  age  17 
mathematics  means  on  each  subscale  to  the  corresponding  averages  of  the  age 
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group  means  across  the  three  subscales.  That  is,  let  the  means,  by  age,  of 
the  initial  subscale  proficiencies  be  as  follows: 

Subscale  Age  9  Age  13  Age  17 

Measurement  6^ ^  ^  9^  f)^ 

1,9  1,13  ^  1,17 


N  6c  0  (H-L) 


N  6c  0  (K  6c  S) 


^°2,9  ^°2,13  ^^,17 

^^,9  ^°3,13  ^^,17 


Further,  let  the  averages  across  the  three  age-spanning  subscales  for  the  age 
9  and  the  age  17  samples  be 

8%    -  +  <?°2,9  ^  '\.9^  /  3  .and 

Then  the  intermediate  transformed  value  for  an  initial  measurement  subscale 
proficiency  value  of  6^^  is 

and  the  intermediate  transformed  values  for  the  remaining  two  age- spanning 
subscales  are  analogously  defined.  Note  that  this  method  permits  means  to 
vary  for  the  age  13  samples. 

For  the  geometry  subscale,  which  appeared  in  only  the  higher  two  age 
groups,   the  age  17  mean  was  matched  to  the  average  of  the  age  17  means  across 
the  three  age  spanning  subscales,  but  the  age  13  mean  was  matched  to  the 
average  transformed  age  13  mean  obtained  in  the  three  mathematics  subscales 
that  spanned  all  three  ages.     That  is,  the  intermediate  transformed  value  for 

an  j.nitial  geometry  subscale  proficiency  value  of  9^^  is 

where  -j^j  and  9^^        are  the  geometry  initial  proficiency  means  for  ages 

13  and  17  and  where 
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is  the  average  of  the  transformed  age  13  mean  values  for  two  subscales-- 
measurement,  numbers  and  operations:  higher-level  and  numbers  and  operations: 
knowledge  and  skills . 

Finally,  for  the  relations  and  functions- -algebra  subscale,  which 
appeared  only  at^age  17,  the  intermediate  transformed  value  for  an  initial 
proficiency  of  ^  ^  was  set  to 

where  9^^        is  the  initial  proficiency  mean  for  age  17  and  where  a  is  the 

5,17  111  1  1 

geometric  mean  of  a  ^,  a  ^,  a  ^,  and  a  ^  where  a  ^  is  the  standard  deviation 

for  the  age  17  sample  for  the  intermediate  transformed  value  of  the  i  ' 
subscale . 

This  method  of  scale  determination  constrains  the  age  9  means  to  be 
equal  across  subscales  and  the  age  17  means  to  be  equal  across  subscales,  but 
the  age  13  means  can  be  expected  to  vary  slightly. 


10.1.5    Definition  of  the  Mathematics  Composite  and  Setting 
the  Final  Origin  and  Unit-Sizes 

Although  examination  of  results  by  subscale  provides  much  useful 
information  about  how  students  perform  on  the  facets  of  mathematics  addressed 
by  the  subscales,  this  level  of  detail  is  too  great  for  many  purposes.  High- 
level  policymakers  and  the  public  at  large  have  the  need  for  a  single  index 
of  performance  in  mathematics,  to  allow  the  summarization  of  overall 
performance.     To  fulfill  this  need,  a  mathematics  composite  was  defined  as  a 
weighted  average  of  the  results  across  subscales.     (The  high  correlations 
between  the  subscales  within  each  of  the  three  grade/age  groups,  shown  in 
Table  10.9,  shows  that  much  information  is  shared  between  the  subscales.) 

The  mathematics  composite  was  defined  separately  for  each  grade/age  as  a 
weighted  average  of  the  estimated  student  proficiencies  (plausible  values) 
for  the  subscales  appearing  in  that  grade/age  (after  the  intermediate 
transformations  of  the  last  section),  with  weights  that  reflect  the  number  of 
items  in  that  subscale  on  the  assessment  for  that  grade/age.     (The  weights 
were  in  fact  proportional  to  the  percentage  distribution  of  items  by  age  and 
content  area  specified  in  Math  Objectives.  1985-86  Assesi  ient,  [NAEP, 
1986a]).     This  is  a  near  optimal  weighting  of  the  subscales  in  terms  of  the 
precision  of  the  resulting  composite  and  is  optimal  in  terms  of  relative 
importance  of  the  subscales  implicit  in  the  specifications  of  the  Learning 
Area  Committee.      The  definition  of  the  composite  in  each  grade/age  is  given 
in  Table  10.10. 

As  described  in  section  10.1.4,  certain  of  the  linear  Indeterminacies  in 
the  subscales  had  been  resolved  by  the  anchoring  of  the  subscale  age  means. 
However,  this  anchoring  still  leaves  an  indeterminacy  in  the  subscales  and  in 


-230- 


Table  10.9 


Estimated  Correlations  between  Subscales 
(Based  on  the  First  Plausible  Value) 


Grade  3/Age  9 

Measurement  N  &  Q  rH-L^  N  &  0  (K  & 

Measurement  1.00  .63  .60 

Numbers  and  Operations: 

liigher- level  Applications  .63  1.00  .60 

Numbers  and  Operations: 

Knowledge  and  Skills  .60  .60  1.00 


Measurement 

Measurement  1 . 00 
Numbers  and  Operations : 

Higher- level  Applications  .73 
Numbers  and  Operations: 

Knowledge  and  Skills  .65 

Geometry  ^  64 


Grade  7/Age  13 
N  &  Q  (H-L^  N  &  0  (K  &  Geometry 


.73 

.65 

.64 

1.00 

.64 

.64 

.64 

l.OO 

.58 

.64 

.58 

1.00 

Grade  11/Age  17 
Measurement  N  &  Q  (H--L^  N  &  0        &        Geometry    R  &  F 

Measurement 

Numbers  and  Operations: 

Higher-level  Applications 
Numbers  and  Operations : 

Knowledge  and  Skills 
Geometry 

Relations  and  Functions 


1-00  .75  .64  .72  .69 

.75  1.00  .68  .74  .74 

.64  .68  1.00  .63  .66 

'72  .74  .63  1.00  .73 

•69  .74  .66  .73  1.00 
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Table  10.10 


Defining  Weights  for  the  Mathematics  Composite 
by  Grade/Age 


Grade  3/ 

Grade 

7/ 

Grade  11/ 

Subscale 

Af;e  9 

Ape 

13 

Age  17 

Measurement 

28 

22 

17 

Geometry 

0 

11 

14 

Relations  c-nd  Functions 

0 

0 

17 

Numbers  and  Operations- - 

Higher-level  Applications 

36 

33. 

5 

26 

Numbers  and  Operations-- 

Knowledge  and  Skills 

36 

33. 

5 

26 

Total 

100 

100 

100 

the  composite  in  that  the  means  and  standard  deviations  of  the  subscale- 
scores  and  hence  the  composite  scores  are  still  arbitrary.     To  resolve  this 
ambiguity,  the  final  step  in  the  creation  of  the  mathematics  subscales  and 
the  composite  scale  was  to  linearly  transform  the  intermediate  composite 
scale  so  that  the  final  composite  would  have  a  weighted  mean  of  250.5  and  a 
weighted  standard  deviation  of  50  across  all  students  in  the  three 
grade/ages.     The  result  is  that  the  overall  mathematics  composite  has  the 
same  mean  and  standard  deviation  as  did  the  1984  reading  proficiency  scale. 

The  same  linear  transformation  which  created  the  final  composite  was 
then  applied  to  each  of  the  intermediate  mathematics  subscales.     Table  10.11 
shows  the  coefficients  of  the  (overall)  linear  transformations  used  to 
transform  the  subscales  from  their  original  units  (on  the  0-1  scale)  to  their 
final  units. 


Table  10.11 
Coefficients  of  the  Linear  Transformations 
of  the  Subscales  from  their  Original  Units 
to  the  Units  of  the  Composite 

Subscale  Intercept  Slope 

Measurement  255.40  51.69 

Geometry  285.42  32.29 

Relations  and  Functions  301.70  35.00 
Numbers  and  Operations: 

Higher- level  Applications  249.52  53.38 
Numbers  and  Operations: 

Knowledge  and  Skills  256.14  54.20 


-232- 

22-; 


The  item  parameters  shown  in  Tables  E.4  through  E.15  of  Appendix  E  and 
the  conditioning  effects  shown  in  Tables  B.4  through  B.9  of  Appendix  B  are  in 
the  metrics  of  the  original  calibration  of  the  subscales.     To  represent  these 
parameters  and  effects  in  terms  of  the  metric  of  the  final  reporting 
subscales,  the  following  transformations  need  to  be  performed.     For  a  given 
subscale,  let  a      b^  and  c^  be  the  item  parameters  and        the  estimated 
conditioning  effects,  expressed  in  terms  of  the  metric  of  the  original 
calibration  scale.     Let  a  and  /3  be  the  intercept  and  slope  (from  Table  10.11) 
of  the  linear  transformation  of  the  subscale  proficiencies  from  their 
original  (calibration)  units  to  their  final  (reporting)  units.     Then,  the 
transformation  of  a  proficiency  $  in  original  units  to  the  metric  of  the 
reporting  scale  is  MP  -  a  +  ^  ^.     The  corresponding  transformations  of  the 
item  parameters  and  estimated  conditioning  effects  are 

^MP  ■  ""0'  ^""^ 

Within  a  grade/age  group,  the  composite  encompasses  the  subscales 
defined  for  that  grade /age,  weighted  in  accordance  with  the  Learning  Area 
Committee's  specifications,  and  consequently  provides  an  index  of  mathematics 
achievement  applicable  to  that  grade/age.     However,  since  the  number  of 
subscales  going  into  the  composite  differs  by  grade/age,  there  is  the 
question  of  the  comparability  of  the  composite  scale -scores  across 
grade/ages.     To  justify  the  inclusion  of  additional  subscales  into  the 
composite  at  the  higher  grade/ages.  Table  10.12  shows  a  comparison  of 
selected  subgroup  means  for  students  in  grades  7  and  11  for  the  mathematics 
composite  as  it  is  defined  with  the  equivalent  means  based  on  a  weighted 
average  of  the  three  age  spanning  subscales.     This  latter  weighted  mean  uses 
weights  proportional  to  the  percentage  distribution  of  items  by  age  and  for 
these  three  subscales  as  specified  in  Math  Objectives.  1985-86  Assessment 
(NAEP,  1986a).    (The  mathematics  composite  :     identical  to  this  weighted  mean 
for  third  grade  students).     It  can  be  seen  from  the  table  that  there  are  only 
trivial  differences  between  the  two  composites  in  terms  of  their  subgroup 
means,  due  to  the  high  degree  of  correlation  between  the  subscales  (Table 
10.9)  and  the  fact  that  the  age  group  means  of  each  of  the  subscales  have 
been  anchored  to  the  same  points.     Consequently,  a  composite  based  on  all 
available  subscales  at  a  given  grade/age  retains  the  advantage  of  summarizing 
over  all  facets  of  mathematics  defined  for  that  grade/age  while  allowing 
meaningful  comparisons  between  the  grade /ages. 

Finally,  it  is  necessary  to  caution  that,  although  the  mathematics 
composite  is  expressed  in  apparently  the  same  units  as  the  1984  reading 
proficiency  scale  in  that  both  scales  have  the  same  means  and  standard 
deviations,  it  is  not  appropriate  to  compare  scores  on  the  mathematics 
composite  with  scores  on  the  reading  scale.    The  transformation  chosen  to 
resolve  the  linear  indeterminacies  in  the  mathematics  composite  is  a 
convenient  transformation,  but  is  only  one  of  a  conceptually  infinite  number 
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Table  10.12 


Comparison  by  Grade  and  Demographic 
Subgroups  of  the  Mathematics  Composite  Mean 
with  the  Mean  of  the  Three  Age-Spanning  Subscales 

Grade  7  Grade  11 


Mean  of  the  Mean  of  the 

Composite         Age -spanning  Composite          Age -spanning 

Mean  Subscales  *  Mean  Subscales 


Total 

267 . 

1 

(0. 

6)** 

267. 

3 

(0. 

6) 

304. 

0 

(0. 

7) 

303. 

9 

(0. 

7) 

Male 

266. 

6 

(0. 

6) 

266. 

6 

(0. 

6) 

306. 

1 

(1. 

0) 

306 . 

5 

(0. 

9) 

Female 

267. 

6 

(0. 

7) 

267. 

9 

(0. 

7) 

301. 

8 

(0. 

8) 

301. 

4 

(0. 

8) 

White 

274. 

0 

(0. 

6) 

274. 

2 

(0. 

7) 

309. 

4 

(0. 

7) 

309 . 

6 

(0. 

6) 

Black 

245. 

4 

(0. 

8) 

245. 

1 

(0. 

9) 

279. 

2 

(1- 

2) 

278. 

6 

(1. 

2) 

Hispanic 

251. 

3 

(1. 

1) 

251. 

0 

(1. 

1) 

285. 

6 

(1. 

5) 

2(55. 

2 

(1. 

6) 

Other 

269. 

0 

(7. 

2) 

269. 

3 

(6. 

9) 

317. 

1 

(6. 

4) 

317 . 

4 

(6 

0) 

NE 

275 

3 

(1. 

2) 

275 

4 

(1. 

2) 

309. 

6 

(1 

5) 

309 

0 

(1. 

4) 

SE 

261 

3 

(1. 

3) 

261 

2 

(1. 

2) 

297. 

2 

(1 

1) 

297 

5 

(1. 

0) 

Central 

270 

6 

(1 

2) 

270 

8 

(1 

1) 

305 

6 

(1 

2) 

305 

9 

(1 

1) 

West 

262 

8 

(1 

3) 

263 

0 

(1 

3) 

302 

3 

(2 

0) 

302 

4 

(1 

8) 

ERural 

265 

.6 

(3 

4) 

265 

.8 

(3 

2) 

299 

0 

(3 

■4) 

300 

.1 

(3 

3) 

LMet 

246 

.4 

(2 

.2) 

246 

.2 

(2 

2) 

279 

9 

(3 

.3) 

279 

.2 

(3 

1) 

HMet 

281 

.9 

(1 

.9) 

282 

.4 

(1 

.9) 

320 

.9 

(2 

.1) 

320 

.3 

(2 

.0) 

<HS 

249 

.4 

(0 

.8) 

249 

.5 

(0 

.9) 

284 

.5 

(1 

.3) 

285 

.0 

(1 

■  4) 

Grad  HS 

260 

.5 

(0 

.6) 

260 

.4 

(0 

.7) 

293 

.8 

(0 

.7) 

294 

.4 

(0 

.8) 

>HS 

275 

.0 

(0 

.7) 

275 

.5 

(0 

.8) 

306 

.6 

(0 

.8) 

306 

.6 

(0 

.8) 

Grad  Col 

278 

.5 

(0 

.9) 

278 

.8 

(0 

.9) 

316 

.0 

(0 

.9) 

315 

.5 

(0 

■9) 

Unknown 

251 

.4 

(1 

.0) 

250 

.9 

(1 

.1) 

278 

.8 

(1 

.2) 

278 

.8 

(1 

.6) 

Public 

265 

.6 

(0 

.5) 

265 

.7 

(0 

.5) 

302 

.7 

(0 

.7) 

302 

.9 

(0 

.7) 

Non-publ 

279 

.8 

(2 

.5) 

280 

.1 

(2 

.5) 

315 

.8 

(2 

.3) 

314 

.6 

(2 

.2) 

*      Weighted  mean  of  the  Measurement,  Numbers  and  Operations:  Higher-level 
Applications,  and  Numbers  and  Operations:     Knowledge  and  Skills  subscales 
with  weights  determined  by  the  Math  Objectives.  1985-86  Assessment  (NAEP, 
1986a). 

Standard  errors  presented  in  parentheses  (the  correlations  between  the 
composite  mean  and  the  mean  of  the  age-spanning  subscales  exceeds  .97). 
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of  such  transformations  that  could  have  been  chosen,  any  one  of  which  would 
have  provided  equivalent  information  about  the  relative  standings  of 
subgroups  of  the  population  in  terms  of  their  abilities  in  mathematics. 


10.1.6    Anchoring  the  Points  on  the  Mathematics  Composite 

One  of  NAEP's  major  goals  has  always  been  to  describe  what  students  know 
and  can  do  and  stimulate  debate  about  whether  those  levels  of  performance  are 
satisfactory.     An  additional  benefit  of  scale-score  methodology  is  that  it 
provides  for  a  criterion-referenced  interpretation  of  levels  on  a  continuum 
of  proficiency.     NAEP  initiated  the  scale  anchoring  process  for  the  1984 
reading  proficiency  scale  and  has  applied  a  technique  in  the  same  spirit  as 
that  scale  anchoring  for  the  anchoring  of  the  mathematics  composite.     In  both 
cases,  the  levels  150,  200,  250,  300,  and  350  on  the  scale  were  chosen  as 
anchor  points.     Each  level  was  defined  by  describing  the  types  of  mathematics 
or  reading  (as  the  case  may  be)  questions  that  most  students  attaining  that 
proficiency  level  would  be  able  to  perform  successfully  and  each  level  was 
exemplified  by  typical  benchmark  items. 

The  difference  between  the  anchoring  of  reading  proficiency  and  the 
anchoring  of  the  mathematics  composite  stems  from  the  fact  that  the  reading 
scale  was  based  on  a  univariate  IRT  model  while  the  mathematics  composite  is 
an  average  of  a  number  of  subscales.     In  the  case  of  reading,  the  IRT  model 
provided,  through  the  item  parameters,  the  probability  of  a  correct  response 
by  a  randomly  selected  pupil  at  any  point  on  the  proficiency  scale.  Since 
these  probabilities  are  not  directly  available  in  the  case  of  the  mathematics 
composite,  they  are  empirically  determined  by  the  proportion  of  the  assessed 
population  at  any  given  level  who  answered  the  item  correctly.  More 
precisely,  the  probability  of  answering  a  particular  item  correctly,  given 
that  the  student's  proficiency  is  at  the  300  level  (for  example),  was 
estimated  by  the  proportion  of  students  answering  the  item  correctly  who  also 
had  composite  proficiencies  within  12.5  units  of  the  target  level  (i.e. 
between  287.5  and  312.5  for  the  300  level).     To  avoid  problems  of  instability 
of  the  estimated  probabilities  for  very  small  numbers  of  respondents  to  an 
item,  the  probability  was  not  defined  if  fewer  than  10  students  at  a  given 
proficiency  level  responded  to  the  item. 

Apart  from  this  difference  in  the  estimation  of  probabilities,  the 
anchoring  of  reading  and  mathematics  proceeded  iu  the  same  way.     Details  of 
the  reading  anchoring  appear  in  Beaton  (1987b). 

In  the  scale-anchoring  process  for  the  mathematics  composite,  NAEP 
identified  sets  of  items  from  the  1986  assessment  that  were  good 
discriminators  between  proficiency  levels.     The  guideline  used  to  select  such 
items  war.  that  students  at  any  given  level  would  have  at  least  a  65  to  80 
percent  (but  often  higher)  probability  of  success  with  these  mathematics 
questions,  while  the  students  at  the  next  lower  level  would  have  a  much  lower 
probability  of  success.     The  criterion  used  was  that  the  difference  in 
probabilities  between  adjacent  levels  should  exceed  30  percent.  Mathematics 
educators  examined  these  empirically  selected  items  sets  and  used  their 
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expert  judgment  to  characterize  each  proficiency  level,  contrasting  tasks  at 
that  level  with  those  at  the  levels  just  above  and  below. 

10. 1.7    WARM  Background  Composites 

In  addition  to  the  mathematics  cognitive  items,  students  at  all  three 
age/grade  levels  were  asked  questions  about  their  coursework,  their  attitudes 
toward  mathematics,  and  the  type  of  instruction  they  had  received.  The 
weighted  average  response  method  (WARM)  was  used  to  construct  sets  of 
composite  background  variables  based  on  background  questions  specific  to 
mathematics  that  were  included  in  the  mathematics  blocks .     A  general 
description  of  the  WARM  procedure  as  it  was  applied  to  the  1986  assessment 
appears  in  Chapter  8 . 

The  questions  comprising  each  of  the  mathematics  WARM  composites  appear 
in  Appendix  C  in  Tables  C.5,  C.6  and  C.7,  for  grades  3,  7,  and  11 
respectively.     The  mapping  of  the  original  responses  to  the  questions  to  the 
scaled  and  oriented  responses  used  for  the  WARM  composites  appears  in  Table 
C.8  of  Appendix  C. 

10.2    SCALING  OF  THE  TREND  DATA 

For  the  portion  of  the  assessment  designed  to  measure  trends,  students 
were  administered  previously  assessed  mathematics  questions  according  to  the 
procedures  used  in  prior  assessments.     Only  students  eligible  by  age  were 
sampled  in  this  trend  assessment.     A  total  pool  of  68  questions  were  given  at 
age  9,  98  at  age  13,  and  94  at  age  17,  with  each  of  the  booklets  accompanied 
by  a  paced  audio  recording  of  the  questions  as  was  done  in  the  first  three 
assessments  of  mathematics.      The  pool  of  items  that  was  scaled  consisted  of 
all  noncalculator  items  from  the  trend  portion  of  the  1986  assessment  which 
were  also  given  in  at  least  one  of  the  1977-78  and  1981-82  assessments  of 
mathematics.     Due  to  the  sparsity  of  trend  items  within  the  individual 
subscales  defined  in  section  10.1,  a  single  scale  was  fit  to  these  trend 
items.     Because  very  few  of  the  items  were  also  given  in  the  1973-74 
mathematics  assessment  (1  at  age  9 ,  2  at  age  13,  4  at  age  17),  the  data  from 
this  assessment  was  not  included  in  the  scaling  for  trend.     (See  section 
10.2.3,  below,  for  the  technique  used  to  extrapolate  1973  results  onto  the 
trend  scale . ) 

The  sample  sizes  for  the  estimation  of  trend  are  as  follows: 


Table  10.13 

Student  Sample  Sizes  for  Mathematics  Trend  Scaling 


1977-78 


1981-82 


1986 


Age 
Age 
Age 


9 

13 

17  (in-school) 


14,752 
24,209 
26,756 


12,038 
15,758 
16,319 


6,932 
6,200 
3,868 
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The  measurement  of  trends  in  mathematics  achievement  over  time  was  based 
on  a  somewhat  different  sample  from  that  used  for  the  1986  spiral  assessment 
results.     In  contrast  to  the  BIB-spiral  administration,  where  students  read 
items  silently  to  themselves  in  timed  blocks,  the  method  of  administration  in 
previous  NAEP  mathematics  assessments  used  tape  recordings  to  read  items  and 
pace  students  through  the  session.     Furthermore,  the  range  of  birthdates 
which  defined  9 -year- old  and  13 -year- old  students  was  different  in  the  BIB- 
spiral  administration  than  in  previous  assessments.     Bridge  samples  of  pace- 
administered  mathematics  items  were  included  in  the  1986  assessment  in  order 
to  enable  comparisons  with  previous  NAEP  assessments.     To  adjust  for  the 
changes  in  age  definition  in  the  case  of  9-  and  13-year~old  students,  two 
separate  bridge  samples  of  tape-recorder-administered  items  were  included  in 
the  assessment,  one  using  the  old  age  definitions  and  one  the  definitions 
used  in  the  BIB-spiral  administration.     The  scaling  for  trends  was  carried 
out  using  the  bridge  data  from  the  1986  assessment  and  data  from  the  NAEP 
mathematics  assessments  in  1977-78  and  1981-82. 

Consequently,  the  scaling  for  trend  included  11  distinct  samples  of 
students,  defined  as  follows: 


Table  10.14 

Samples  of  Students  Used  in  the  Scaling  for  Trend 


Assessment 

Age  9 

Aee  13 

1977-78 

X 

X 

X 

1981-82 

X 

X 

X 

1986  Bridge  A 

X 

X 

1986  Bridge  B 

X 

X 

X 

In  the  above.  Bridge  A  refers  to  the  tape -recorder- administered  assessment  of 
9-  and  13 -year- olds  using  the  old  age  definitions  and  time  of  assessment; 
Bridge  B  refers  to  the  tape-recorder-administered  assessment  of  9-,  13-,  and 
17 -year- olds  using  the  new  age  definitions  and  time  of  assessment.  (The 
Bridge  B  assessment  of  17-year-olds  actually  uses  the  same  age  definitions 
and  time  of  assessment  as  in  previous  assessments,  since  these  were  unchanged 
for  17-year-olds). 


10.2.1    Estimation  of  Item  Parameters  and  Generation  of  Plausible  Values 

As  noted  above,  there  were  insufficient  items  per  content  area  to 
support  the  creation  of  subscales .     Accordingly,  the  scaling  for  trends  in 
mathematics  was  accomplished  by  fitting  a  univariate  IRT  model  to  the  set  of 
trend  items. 

The  first  step  in  the  scaling  process  was  the  estimation  of  item 
parameters  for  the  trend  items.     This  item  calibration  was  performed 
separately  for  each  of  the  three  age  groups,  using  data  from  the  last  three 
assessments.     The  calibration  for  each  age  used  the  BILOG  program  and  was 
performed  on  a  subsample  of  all  the  available  subjects,  resulting  in 
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approximately  550  examinees  in  each  assessment  year  for  each  item. 
(Approximately  275  suudents  from  the  Bridge  A  sample  and  275  students  from 
the  Bridge  B  sample  entered  the  calibration  for  each  item) . 

A  total  of  56  items  for  age  9,  79  items  for  age  13,  and  74  items  for  age 
17  were  initially  calibrated.  In  the  course  of  calibration,  item  fit  was 
evaluated  by  inspecting  residuals  from  the  fitted  item  response  curves.     As  a 
result  of  these  examinations,  two  items  were  identified  as  displaying  poor 
fit  and  dropped  from  the  final  scaling.     One  item  (N266801)  was  dropped  from 
the  age  17  scaling  because  of  poor  fit  for  the  1981-82  sample  and  one  item 
(N252601)  was  dropped  from  the  age  9  scaling  because  of  poor  fit  for  both  the 
1981-82  and  the  1986  samples.     An  additional  item  (N286102)  was  dropped  from 
the  age  9  scaling  because  of  a  change  in  presentation  format  which 
accompanied  an  outlying  trend  in  percent  correct  across  the  assessment  years. 

The  results  of  the  item  calibration  phase  were  item  parameters  for  54 
items  for  age  9,  79  items  for  age  13,  and  74  items  for  age  17.     Table  10.15 
shows  a  breakdown  of  the  items  scaled  by  age,  assessment  year,  and  content 
category.     A  list  of  the  items  scaled  for  the  three  ages,  along  with  their 
item  parameters  appears  in  Tables  E.31  through  E.33  in  Appendix  E. 

The  next  step  in  the  scaling  of  mathematics  trend  was  the  generation  of 
plausible  values  for  the  trend  scale.     The  estimation  of  plausible  values  was 
conducted  independently  by  age  for  each  of  the  assessment  samples  identified 
in  Table  10.14.       Because  there  were  fewer  background  variables  available  for 
trend,  there  were  fewer  conditioning  variables  used  in  the  creation  of  the 
plausible  values.     The  conditioning  variables  and  the  estimated  conditioning 
effects  are  given  in  Tables  B.IO  through  B.20  in  Appendix  B  for  the  three 
grade/age  groups . 


10.2.2    Linking  to  the  Cross-sectional  Mathematics  Composite 

The  units  of  the  final  trend  scale  were  determined  linking  each  of  the 
three  age  scales  to  the  mathematics  composite  scale.     This  was  done  for  each 
age  by  matching  the  mean  and  standard  deviation  of  on  the  trend  scale  of  the 
1986  Bridge  B  sample  (the  sample  with  the  new  age  definition)  to  the  mean  and 
standard  deviation  on  the  composite  mathematics  scale  of  the  corresponding 
age  sample.     Table  10.16  shows  the  coefficients  of  the  linear  transformations 
used  to  transform  the  trend  scale.     A  transformation  analogous  to  that 
discusses  in  section  10.1.5  can  be  used  to  transform  the  item  parameters 
presented  in  Appendix  E  and  estimated  conditioning  effects  presented  in 
Appendix  B  from  the  original  calibration  metric  to  the  final  reporting 
metric. 
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Table  10.15 
Counts  of  Mathematics  Trend  Items 
by  Age,  Year,  and  Content  Area 


L/e±  L.  d 

Rln.  , 

Year 

Fund. 

Func . 

N  &  0 

N  &  0 

Age 

Meth 

Int. 

Me  as . 

Geom. 

&  Alg. 

HL 

KS 

Total 

1977-78 

9 

A 

o 
o 

1 

1 

2 

3 

11 

36 

13 

A 
*+ 

7 

3 

6 

20 

54 

17 

5 

6 

6 

0 

16 

61 

1981-82 

9 

4 

11 

12 

1 

2 

5 

19 

54 

13 

4 

10 

13 

7 

3 

12 

29 

78 

17 

5 

6 

7 

11 

9 

14 

20 

72 

1986 

9 

4 

11 

12 

1 

2 

5 

19 

54 

13 

4 

10 

13 

7 

3 

12 

29 

78 

17 

5 

6 

7 

11 

10 

14 

20 

73 

Table  10.16 
Coefficients  of  the  Linear  Transformation 
of  the  Trend  Scale  from  Original  Units 
to  Units  of  the  Composite 

Age  Intercept  slope 

9  218.42  35.84 

13  266.58  34.57 

17  300.70  33.88 


10.2.3    Extrapolation  of  the  1973-74  Mean  P-value  Results  onto  the  Trend 
Scale 

Because  of  insufficient  items  in  coimnon  with  the  1986  trend  assessment, 
the  1973-74  mathematics  assessment  was  not  included  in  the  scaling  of  NAEP 
trend  data.     However,  for  the  nation  and  several  reporting  subgroups  (e.g 
male,  female)  at  each  of  the  three  age  levels,  an  estimate  of  the  1973-74 
mean  level  of  student  mathematics  proficiency  was  computed. 

These  estimates  were  obtained  by  assuming  that  the  relationship  within  a 
given  age  level  between  the  logit  of  a  subgroup's  mean  p-value  (i.e.,  mean 
proportion  correct)  and  its  respective  mathematics  proficiency  mean  was 
linear  and  that  the  same  line  held  for  all  assessment  years  and  for  all 
subgroups  within  the  age  level.     Under  this  assumption,  the  between-year 
difference  of  the  mean  proficiency  values  of  a  subgroup  for  a  pair  of 
assessment  years  is  equal  to  a  constant  (B)  times  the  between-year  difference 
of  the  logits  of  the  mean  p-values  of  that  subgroup  for  the  same  two  years. 
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For  each  age  level,  a  mean  p-value  estimate  using  a  common  set  of  items  was 
available  for  1973-74,  1977-78,  and  1981-82.     The  constant  B  was  estimated  by 
a  regression  (through  the  origin)  of  the  difference  between  proficiency  means 
in  1977-78  and  1981-82  on  the  corresponding  difference  between  the  logits  of 
the  mean  p-values  for  these  two  years.     All  subgroups  in  a  given  age  were 
included  in  the  regression.     The  estimate  of  the  1973-74  proficiency  mean  for 
a  subgroup  was  then  obtained  as  the  sum  of  the  1977-78  subgroup  mean 
proficiency  and  B  times  the  difference  between  the  logits  of  the  1973-74  and 
1977-78  subgroup  mean  p-values. 

The  quality  of  this  extrapolation  technique  was  evaluated  by  comparing 
its  performance  in  predicting  the  1977-78  data.     The  actual  values  of  the 
1977-78  subgroup  mean  proficiencies  were  compared  with  the  predicted  values 
formed  as  the  sum  of  the  1981-82  subgroup  mean  proficiency  and  B  times  the 
difference  between  the  logits  of  the  1977-78  and  1981-82  subgroup  mean  p- 
values.     The  predictions  were  very  close  to  the  actual  values,  the  residual 
me^-is  squared  error  being  only  .4  percent  of  the  variance  of  the  actual 
V  *lues . 
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Chapter  11 
SCIENCE  DATA  ANALYSIS 


Kentaro  Yamamoto 
Educational  Testing  Service 


The  design  for  the  analysis  of  the  spiral  and  bridge  sample  data  in  the 
science  assessment  was  similar  to  the  one  developed  for  the  mathematics  data 
(Chapter  10).     The  19S6  science  assessment,  reported  in  The  Science  Report 

Q^ld:  Elements  of  Risk  and  Recovery  (Mullis  &  Jenkins ,   1988),  examined  two 

different  aspects  of  evaluating  proficiency  in  science.     One  was  the  cross - 
sectional  data  analysis,  undertaken  to  examine  how  subgroups  of  the 
population  compare  to  each  other  across  ages.     The  other  was  trend  data 
analysis,  which  examined  how  1986  student  achievement  compared  to  past 
assessments,  hence  permitting  the  examination  of  changes  in  science 
proficiency  over  time.     This  chapter  describes  the  technical  details  of  the 
item  parameter  estimation,  and  rescaling  performed  for  the  science  cross - 
sectional  and  trend  analyses.     The  underlying  theory  is  discussed  in  Chapter 


11.1     CROSS -SECTIONAL  DATA  ANALYSIS 

The  design  for  the  analysis  of  the  spiral  sample  data  in  the  science 
assessment  involved  three  different  grade/age  groups.     In  addition,  two 
bridge  samples  were  used  to  link  the  trend  data  to  the  BIB  data.     All  samp 
were  multistage  stratified  random  samples  across  the  nation  (see  Chapter  3 
for  a  discussion).     Sample  sizes  are  listed  in  the  following  table. 

Table  11.1 

Science;  Scale  Item  Information,  by  Sample 


Sample  No,   of  Cases  No.  of  Items 

Grade  3/Age  9,  Spiral  11,575  124 

Grade  3/Age  9,  Bridge  A  6,932  56 

Grade  3/Age  9,  Bridge  B  4,042  56 

Grade  7/Age  13,  Spiral  11,171  194 

Grade  7/Age  13,  Bridge  A  6,200  76 

Grade  7/Age  13,  Bridge  B  4,178  76 

Grade  11/Age  17,  Spiral  20,535  244 

Grade  11/Age  17,  Bridge  B  3,868  78 


David  Freund,  Bruce  Kaplan,  Maxine  Kingston,  Edward  Kulick,  Michael 
Narcowich,  Jennifer  Nelson,  Alfred  Rogers,  and  Minhwei  Wang  performed  the 
data  analysis.     Revert  Mislevy,  Kathleen  Sheehan,  and  Eugene  Johnson  provided 
consultation,     Janet  Johnson  provided  valuable  editorial  assistance. 
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11.1.1    Definition  of  Subscales 


A  total  of  447  distinct  science  items  were  administered  in  the  1986 
spiral  assessment  using  a  BIB-spiral  design  (Messick,  Beaton,  &  Lord,  1983; 
Beaton,  1987a)  to  allocate  the  items  to  the  sampled  students  (Chapter  3). 
Seven  blocks  of  science  questions  were  assessed  at  grade  3/age  9,  9  blocks  at 
grade  7/age  13,  and  11  blocks  at  grade  11/age  17.     Two  of  the  grade  7/age  13 
blocks  (nxjunbers  5  and  6)  were  also  used  at  grade  11/age  17.     Out  of  52 
booklets  prepared  for  grade  3/age  9,  30  contained  one  or  more  science  blocks. 
Thirty- two  of  68  booklets  for  grade  7/age  13  and  42  of  93  booklets  for  grade 
11/age  17  contained  at  least  one  science  block.     Detail'^ d  arrangements  of 
blocks  and  items  contained  in  them  are  docximented  in  chapter  4. 

The  pool  of  items  used  in  the  1986  science  assessment  contained  a  range 
of  opea-ended  and  multiple-choice  items.     The  framework  for  the  development 
of  the  items  was  formulated  by  nationally  representative  panels  of  science 
specialists,  educators,  and  concerned  citizens  and  is  documented  in  Science 
Objectives,  1985-1986  Assessment  (NAEP,  1986b).     The  objectives  were  defined 
in  three  dimensions:     content,  context,  and  cognition.     The  content 
categories  included: 

1)  life  science; 

2)  physics; 

3)  chemistry; 

4)  earth  and  space  science; 

5)  history  of  science;  and 

6)  nature  of  science. 

The  context  categories  included: 

1)  scientific; 

2)  personal; 

3)  societal;  and 

4)  technological. 

The  cognition  categories  included: 

1)  knows; 

2)  uses;  and 

3)  integrates. 

Science  analyses  were  carried  out  along  six  subscales.  Science 
subscales  were  designed  to  capture  the  essential  subdivisions  of  science  as 
indicated  by  the  Science  Learning  Area  Committee  in  order  to  allow  the 
detection  of  potential  differences  in  performance  patterns  between  those 
subdivisions.     The  initial  basis  for  the  definition  of  the  subscales  comes 
from  the  three-dimensional  framework  defined  by  the  Learning  Area  Committee. 
While  this  three-dimensional  framework  is  meaningful  and  finely  defines  the 
subareas  of  interest  to  science  educators,  the  number  of  items  contained  in 
each  category  was  insufficient  to  scale  categories  with  current  technology. 
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As  a  result,  only  the  content  categories  were  used  to  create  subscales. 
Physics  and  chemistry  for  grade  3/age  9  were  combined  and  renamed  physical 
science,  a  term  that  more  aptly  reflects  the  content  of  laarning  for  that 
grade/age.     Also,  since  there  were  only  four  items  in  the  physical  science 
subscale  common  to  the  other  two  grade/age  levels,  including  them  with  the 
items  administered  to  grade  7/age  13  and  grade  11/age  17  would  not  have  been 
appropriate.     The  history  of  science  subscale  was  dropped  because  it  had  an 
insufficient  number  of  items  (25)  to  ensure  accurate  scaling.     Earth  and 
space  science  was  dropped  for  grade  3/age  9  for  two  reasons:    There  were  only 
13  earth  and  space  science  items  for  grade  3/age  9,  and  there  were  only  two 
items  common  to  other  grade/ages. 


11.1.2    Estimation  of  Item  Parameters  for  the  Subscales 

The  computer  program  BILOG  (Mislevy  &  Bock,  1982)  was  used  to  estimate 
the  item^ parameters  of  the  three  parameter  IRT  model  for  the  items  in  each 
subscale  independently  using  roughly  a  one-fourth  sample  of  the  1986  sample. 
Eight  items  were  responded  to  by  slightly  fewer  than  1,000  subjects;  all 
other  items  were  responded  to  by  at  least  1,000  subjects.     The  largest 
response  size  was  over  3,000.     (See  Beaton,  1987a,  for  further  description  of 
the  calibration  process.) 

Table  11.2  lists  the  names  of  the  subscales  and  ntunbers  of  items  for 
each  subscale  for  each  grade/age  group  after  scaling.     For  various  reasons 
described  later  in  this  chapter,  some  items  were  dropped  from  the  data 
analysis.     A  total  of  396  items  was  used  to  analyze  the  1986  data;  384  were 
multiple-choice  and  12  were  open-ended.     Among  multiple-choice  items,  66  had 
only  two  choices.     Consequently  these  66  items  had  low  discrimination 
parameter  values  with  high  "guessing"  parameter  values  (about  .50).  It 
should  be  noted  that  at  the  time  of  calibration,  14  items  in  the  earth  and 
space  science  subscale  for  grade  3/age  9  were  included,  but  later  12  items 
given  only  to  grade  3/age  9  were  dropped.     The  inclusion  of  these  12  items 
had  little  impact  on  the  calibration  of  remaining  science  items,  because  only 
two  items  were  common  to  other  grade/ages.     Hence,  a  linkage  of  ability 
distribution  of  grade  3/age  9  to  the  other  grade/ages  is  very  weak. 

Items  were  dropped  from  other  subscales  because  of  a  lack  of  fit  to  the 
IRT  model.     Items  that  were  dropped  from  the  item  pool  often  did  not  ha/e 
monotone  increasing  empirical  item  characteristic  curves.     There  were  no 
specific  characteristics  that  were  common  to  these  dropped  items.     Table  11.3 
lists  the  items  that  are  dropped  because  of  lack  of  fit. 
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Table  11.2 

Identification  of  Science  Sulncales  and  Nximber  of  Items  Used  for  Analysis 


Number  of  Items 


Grade  3/    Grade  7/    Grade  11/ 


Subscale 

Total 

Dropped 

Age  9 

Age  13 

Age  17 

Life  Science 

116 

(4) 

39 

44 

59 

Chemistry 

55 

(1) 

23 

44 

Nature  of  Science 

67 

(4) 

17 

33 

36 

Physics 

62 

(2) 

30 

44 

Earth  c.nd  Space  Science 

52 

(12) 

42 

39 

Physical  Science 

44 

(3) 

44 

History  of  Science 

0 

(25) 

Total 

296 

(51) 

100 

172 

222 

Table  11,3 

Science  Items  Dropped  for  Specified  Populations 
Because  of  Lack  of  Fit  to  IRT 


Subscale 

Subscale 

Age  Group 

Age  Group 

NAEP  ID 

Ntimber 

Name 

Deleted 

Remaining 

400201 

1 

Life  Science 

9 

13,17 

401701 

5 

Physical  Science 

9 

402004 

6 

Physical  Science 

9 

402601 

6 

Physical  Science 

9 

404903 

3 

Chemistry 

13 

407501 

4 

Physics 

13 

409401 

3 

Nature  of  Science 

13 

409402 

3 

Nature  of  Science 

13 

409403 

3 

Nature  of  Science 

13 

410002 

1 

Life  Science 

17 

412901 

5 

Earth  &  Space  Science 

9 

13 

413301 

3 

Nature  of  Science 

9,13 

414001 

1 

Life  Science 

13 

430101 

1 

Life  Science 

17 

431501 

4 

Physics 

17 

437201 

1 

Life  Science 

17 

Tables  E,16  through  E.28  in  Appendix  E  show  the  estimated  item 
parameters ,     These  item  parameter  estimates  are  from  the  BILOG  computer 
program  prior  to  any  rescaling.     The  unit  and  origin  were  set  provisionally 
in  each  subscale  calibration  run  by  standardizing  the  distribution  of  the 
calibration  sample  of  examinees.     One,  two,  or  three  age  groups  were  used  to 
calibrate  the  item  parameters  depending  upon  the  subscale  (see  Table  11,2), 
Therefore,  item  parameters  in  different  subscales  are  not  on  the  same  metric, 
and  are  not  directly  comparable.     Items  in  different  subscales  should  not  be 
recombined  to  form  new  subsets  of  items  for  any  reason  whatever  without  first 
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recalibrating  the  item  parameters  for  the  set  of  items  the  user  intends  to 
group . 


11.1.3    Generation  of  Multivariate  Plausible  Values  for 
the  Subscales  and  for  Rescaling 

Multivariate  plausible  values  were  generated  using  the  M-GROUP  computer 
program  (Sheehan,  1985)  according  to  the  procedure  described  in  Chapter  8. 
Initially  each  subscale  was  run  separately  to  obtain  subscale  conditioning 
coefficients  and  error  variances. ,  then  combined  as  the  initial  values  for  the 
multivariate  case.    At  this  point,  it  was  decided  to  delete  the  earth  and 
space  science  subscale  for  age  9;  consequently,  the  multivariate  M-GROUP  was 
repeated  in  order  to  exclude  this  subgroup.     The  final  conditioning 
coefficients  and  error  variances  are  listed  in  Tables  B.21  through  B.26  of 
Appendix  B.     For  the  purpose  of  verification,  the  means  and  standard 
deviations  of  the  multivariate  plausible  values  were  compared  to  the 
univariate  calibrated  ability  distributions  from  the  BILOG  runs.     Note  that 
the  calibrating  sample  is  roughly  a  quarter  of  the  total  sample  that  was  used 
for  the  multivariate  plausible  values;  therefore,  similar  but  not  exact 
values  were  expected.     The  results  are  listed  in  Table  11.4. 

In  Tr.ble  11.4,  the  means  and  standard  deviations  of  the  plausible  values 
for  the  physics  subscale  for  grade  7/age  13  appear  to  be  different  from  the 
values  from  the  BILOG  scale.     In  order  to  determine  that  this  deviation  is 
caused  by  the  selection  of  the  quarter-sample  used  for  calibration,  a  more 
detailed  posterior  population  distribution  was  estimated  and  was  found  to 
have  a  mean  of  -.32  and  standard  deviation  of  .85.     This  is  much  closer  to 
the  multivariate  plausible  value  results  than  to  the  BILOG  results. 
Therefore,  this  strongly  indicated  that  the  difference  was  due  to  sample 
variances  rather  than  to  the  misfit  of  the  measurement  model. 

Even  though  item  parameters  were  estimated  with  the  same  standardization 
method  so  that  the  calibration  sample  of  examinees  have  a  mean  of  0.0  and  a 
standard  deviation  of  1.0,  the  differences  in  the  age  samples  among  subscales 
causes  the  estimated  item  parameters  to  be  on  different  scales.  Consequently 
the  estimated  abilities  of  students  on  all  subscales  have  different  origins 
and  accompanying  standard  deviations.     Since  any  linear  transformation  of  the 
IRT  theta  scale  retains  all  information,   it  is  more  convenient  to  have  all 
subscales  on  a  comparable  scale.     As  with  mathematics,  rescaling  was  carried 
out  in  two  steps.     The  first  step  was  to  put  each  subscale  onto  the  same  unit 
of  measure  as  the  others.     The  second  step  was  to  rescale  the  unit  of  measure 
itself  so  that  the  distribution  of  the  three  age  groups  combined  had  a 
weighted  mean  of  250.50  and  a  weighted  standard  deviation  of  50.0. 

The  method  applied  to  rescale  science  plausible  values  is  essentially 
the  same  as  the  one  used  in  mathematics.     In  science  there  are  two  subscales 
that  span  all  three  grade/ages,  and  an  intermediate  transformation  was 
accomplished  by  matching  the  science  age  9  and  age  17  means  on  each  subscale 
to  the  corresponding  averages  of  the  age  group  means  between  the  two 
subscales.     This  produces  two  different  means  for  age  13  on  two  of  the 
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Comparison  of  the  Subscale  Means  and  Standard  Deviations 
Based  on  the  Multivariate  Plausible  Values 
with  the  Univariate  BILOG  Results 


First  Plausible 
Values 


BILOG 


km/  \JLL^  O  \^  CL  JL. 

Mean 

S.D. 

Mean 

S.D. 

.  3/Age  9 

Life  Science 

-.69 

.81 

-.75 

.79 

Nature  of  Science 

-  .83 

.91 

- .  63 

OA 

.  90 

PhvsicAl  Science 

.08 

1.07 

.00 

1.00 

i  7 /Ape  13 

Life  Science 

.04 

.82 

.00 

.80 

Chemistry 

-.43 

.87 

-.44 

.80 

Nature  of  Science 

-.01 

.76 

-.06 

.79 

Physics 

-.29 

.89 

-.40 

.91 

Earth  &  Space  Science 

-.16 

.94 

-.24 

.91 

B  11/ Age  17 

Life  Science 

.80 

.79 

.75 

.78 

Chemistry 

.50 

1.02 

.44 

.99 

Nature  of  Science 

.73 

.94 

.69 

.84 

Physics 

.39 

.94 

.40 

.92 

Earth  &  Space  Science 

.65 

.91 

.57 

.93 
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subscales  (life  science,  and  nature  of  science).     Chemistry,  physics,  and 
earth  and  space  science  all  appeared  at  the  two  higher  age  groups.     The  age 
17  mean  was  matched  to  the  mean  of  the  age  17  means  across  the  two  age 
spanning  subscales,  but  the  age  13  mean  was  matched  to  the  mean  transformed 
age  13  mean  obtained  from  the  two  science  subscales  that  spanned  all  three 
ages.     Physical  science  appeared  only  at  age  9  and  this  mean  was  set  to  the 
mean  of  the  age  9  subscale  means  (again  over  the  three  age -spanning 
subscales)  and  the  standard  deviation  was  set  to  the  geometric  mean  of  the 
age  9  standard  deviations.     This  method  of  scale  determination  constrains  the 
age  9  mean  to  be  equal  across  subscales  and  the  age  17  means  to  be  equal 
across  subscales,  but  the  age  13  means  can  be  expected  to  vary  slightly. 

11.1.4    Creating  the  Composite  Scale  and  Final  Proficiency  Scale 

While  multiple  proficiency  scales  provide  useful  and  very  revealing 
information  about  the  relative  relationships  among  subpopulations ,   the  desire 
to  have  a  single  index  to  summarize  overall  performance  remains  a  high 
priority  among  policymakers  and  the  public  at  large.     For  that  reason,  a 
science  composite  was  defined  as  a  weighted  average  of  the  results  across 
subscales.     Not  all  subscales  apply  to  all  ages  nor  does  the  importance 
associated  with  each  subscale  remain  the  same  across  all  ages.  Therefore, 
the  weights  assigned  to  compute  the  average  of  the  estimated  subscale 
proficiencies  differ  by  grade/age.     The  weights  were  assigned  proportional  to 
the  percentage  distribution  of  items  by  age  and  content  specified  in  Science 
Objectives,   1985-86  Assessment  (NAEP,   1986b).     This  is  a  nearly  optimal 
weighting  procedure  of  the  subscales  in  terms  of  the  precision  of  the 
resulting  composite.     This  procedure  is  also  optimal  in  terms  of  retaining 
the  relative  importance  of  the  subscales  implicit  in  the  specifications  of 
the  science  Learning  Area  Committee.     The  definition  of  weights  for  the 
composite  in  each  grade/age  is  given  in  Table  11.5. 


Table  11.5 

Defining  Weights  for  the  Science  Composite 
by  Grade/Age 


Grade  3 /Age  9      Grade  7 /Age  13      Crade  11 /Age  17 


Life  Science 

47 

27 

26 

Chemistry 

0 

17 

26 

Nature  of  Science 

33 

17 

16 

Physics 

0 

17 

21 

Earth  &  Space  Science 

0 

22 

16 

Physical  Science 

20 

0 

0 

Total 

100 

100 

100 

As  described  earlier,  the  linear  indeterminacies  among  the  subscales  had 
been  resolved  by  the  anchoring  of  the  subscale  age  means;  that  is,  all  the 
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subscales  are  now  on  the  same  common  scale.     However,  this  still  leaves  the 
indeterminacy  of  the  common  scale  itself.     To  resolve  this  ambiguity,  the 
final  step  in  the  creation  of  the  science  scale  was  to  linearly  transform  the 
intermediate  composite  scale  so  that  the  final  composite  would  have  a 
weighted  mean  of  250.5  and  a  weighted  standard  deviation  of  50  across  all 
students  in  the  three  ages.     The  result  is  that  the  overall  science  composite 
has  the  same  mean  and  standard  deviation  as  did  the  1984  reading  proficiency 
scale. 

The  same  linear  transformation  that  created  the  final  composite  was  then 
applied        each  of  the  intermediate  science  subscales.     Table  11.6  shows  the 
coefficients  of  the  (overall)  linear  transformations  used  to  transform  the 
subscales  from  their  original  units  (calibrating  scale)  to  the  final 
proficiency  scale. 

Table  11.6 

Coefficients  of  the  Linear  Transformations 
of  the  Subscales  from  the  Calibrating  Scale 
to  the  Units  of  the  Reporting  Proficiency  Scale 


Subscale 

Intercept 

Slope 

Life  Science 

244.20 

52.18 

Chemistry 

264.57 

44.34 

Nature  of  Science 

243.63 

56.55 

Physics 

263.12 

60.27 

Earth  &  Space  Science 

253.07 

50.63 

Physical  Science 

203.82 

43.57 

Item  parameters  on  the  calibrating  scale  may  be  transformed  using  the 
above  intercepts  and  slopes  so  that  conditional  probability  of  correct 
response  given  a  proficiency  scale  can  be  obtained.     They  are: 
a(prof iciency)'-a(calibrated)/slope,  b (prof iciency)=slope*b (calibrated) ,  and  c 
parameters  remain  unchanged. 

Finally,   it  is  necessary  to  caution  that,  although  the  science  composite 
is  seemingly  expressed  in  the  same  units  as  the  1984  reading  scale  and  the 
1986  mathematics  scale,   it  is  not  appropriate  to  compare  scores  on  the 
science  composite  with  scores  on  the  other  subject  area  scales.  The 
transformation  chosen  to  resolve  the  linear  indeterminacies  in  the  science 
composite  is  a  convenient  transformation,  but  is  only  one  of  a  conceptually 
infinite  number  of  such  transformations  that  could  have  been  chosen,  any  one 
of  which  would  have  provided  equivalent  information  about  the  relative 
standings  of  subgroups  of  the  population  in  terms  of  their  abilities  in 
science.     There  was  no  link,  real  or  implied,  in  the  construction  of  the 
science  composite  and  the  science  subscales  to  mathematics  or  to  reading. 


-250- 


241 


11.1.5    Anchoring  the  Points  on  the  Science  Composite 

Behavioral  anchoring  was  devised  to  associate  descriptive  statements  of 
a  student's  ability  with  a  level  on  a  continuum  of  proficiency.     This  was 
done  successfully  with  the  1984  reading  scale  and  the  1986  mathematics  scale. 
The  same  technique  was  applied  to  the  science  composite.     As  witn  the  other 
subject  areas,  five  levels- -150,  200,  250,  300,  and  350- -were  selected  on  the 
scale  and  chosen  as  anchor  points.     Each  level  was  defined  by  a  description 
of  the  types  of  questions  that  most  students  attaining  that  proficiency  level 
would  be  able  to  answer  correctly  while  most  students  at  least  one  level 
lower  would  answer  incorrectly.     In  this  way  each  level  was  exemplified  by 
typical  benchmark  items  that  describe  a  subset  of  abilities  indicative  of 
that  level  of  proficiency. 

There  was  no  difference  in  the  anchoring  procedures  used  for  science  or 
mathematics;  both  subjects  used  composite  proficiency  scores  that  were 
derived  from  multivariate  subscale  proficiencies.     For  that  reason  the 
empirical  proportion  correct  was  used  for  science  as  well  as  mathematics  in 
place  of  the  IRT-driven  theoretical  conditional  probabilities.     The  empirical 
proportion  correct  was  calculated  by  selecting  subjects  in  a  range  in  which 
we  were  interested  and  these  responses  were  averaged.     The  ranges  were  set  as 
within  12.5  units  of  the  anchoring  levels.     For  example,  students  who  scored 
between  287.5  and  312.5,  and  also  were  administered  a  particular  item,  were 
used  to  estimate  the  conditional  probability  of  the  correct  response  on  that 
item.     To  avoid  problems  of  instability  of  the  estimated  probabilities  for 
very  small  numbers  of  respondents  to  an  item,  the  probability  was  not  defined 
if  fewer  than  10  students  at  a  given  proficiency  level  responded  to  the  item. 
For  the  details  of  behavioral  anchoring,  see  Beaton  (1987b). 

In  the  scale- anchoring  process  for  the  science  composite,  NAEP 
identified  sets  of  items  from  the  1986  assessment  that  were  good 
discriminators  between  proficiency  levels.     The  guideline  used  to  select  such 
items  was  that  students  at  any  given  level  would  have  at  least  a  65  to  80 
percent  (but  often  higher)  probability  of  success  with  these  science 
questions,  while  the  students  at  the  next  lower  level  would  have  a  much  lower 
probability  of  success.     The  criterion  used  was  that  the  difference  in 
probabilities  between  adjacent  levels  should  exceed  30  percent.  Science 
educators  examined  these  sets  of  empirically  selected  items  and  used  their 
expert  judgment  to  characterize  each  proficiency  level,  contrasting  tasks  at 
that  level  with  those  at  the  levels  just  above  and  below. 


11.1.6    WARM  Background  Composites 

In  a  fashion  identical  to  that  used  for  the  mathematics  assessment, 
students  at  all  three  grade/age  levels  were  asked  questions  about  their 
coursework,  their  attitudes  toward  science,  and  the  type  of  instruction  they 
had  received,   in  addition  to  science  cognitive  items.     The  weighted  average 
response  method  (WARM)  was  used  to  construct  sets  of  composite  background 
variables  based  on  background  questions  specific  to  science  that  were 
included  in  the  science  blocks.     A  general  description  of  the  WARM  procedure 
as  it  was  applied  to  the  1986  assessment  appears  in  Chapter  8.     The  questions 
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comprising  each  of  the  science  WARM  composites  appear  in  Appendix  C  in  Tables 
C.9,  C.IO,  and  C.ll  for  grades  3,  7,  and  11  respectively.     The  matrix  used  to 
map  the  original  responses  to  the  scaled  and  oriented  responses  used  for  the 
WARM  composites  appears  in  Table  C.12, 


11.2    SCALING  OF  THE  TREND  DATA 

To  maintain  continuity  with  the  past  data,  sample  age  definition,  the 
mode  of  delivery  of  items,  and  time  of  assessment  for  trend  data  were 
different  from  cross -sectional  data.     The  trend  data  comprised  subsamples 
defined  by  age  only  (9-,  13-,  and  17 -year-olds) ,  while  the  cross-sectional 
data  included  students  in  the  appropriate  grades  as  well  as  ages  for  all 
three  subsamples.     There  were  three  booklets  used  to  measure  trend  for  ages  9 
and  13.     Each  booklet  contained  a  reading,  a  mathematics,  and  a  science 
block.     Each  student  took  one  of  these  booklets.     The  mathematics  and  science 
parts  of  the  booklets  were  presented  aurally  using  a  tape  recorder  as  in  the 
past  assessments.     The  tape  recorder  was  turned  off  for  the  reading  block. 
There  were  two  booklets  used  to  measure  trend  for  age  17,     Each  booklet 
covered  mathematics  and  science.     These  booklets  were  presented  aurally  to 
the  students  (see  Chapter  4).     There  was  a  total  of  63  items  in  the  science 
trend  item  pool  for  age  9  administered  to  a  total  of  6,932  students;   a  total 
of  83  items  for  age  13  administered  to  a  total  of  6,200  students;  and,  a 
total  of  82  items  for  age  17  administered  to  a  total  of  3,868  students.  The 
science  trend  item  pool  consisted  of  items  that  were  given  in  at  least  one 
previous  assessment.     The  trend  item  pool  was  a  subset  of  the  cross-sectional 
item  pool. 

The  trend  data  analysis  examined  data  from  three  points  in  time,  namely 
the  1976-77,  1981-82,  and  1986  assessments.     Due  to  the  sparsity  of  trend 
items  within  the  subscales  defined  in  section  11.2,  a  single  scale  was  fit  to 
these  trend  items.     Too  few  common  items  between  the  1973-74  and  1986  science 
assessments  prohibited  the  inclusion  of  the  1973-74  data  in  the  trend 
analysis . 

Three  differences  between  trend  data  and  cross-sectional  data  make  them 
incomparable  to  each  other  without  equating.     These  differences  are  mode  of 
administration,  age  definition  of  the  sample,  and  time  of  testing.  The 
bridge  sample  provides  equating  information  by  preserving  the  same  mode  of 
administration  and  age  definition  of  the  sample  as  the  past  trend  data,  and 
the  same  time  of  testing  as  the  cross -sectional  data.     To  align  the  trend  to 
the  cross-sectional,  age  only  students  were  selected  from  the  cross- 
sectional  sample  to  provide  comparable  samples  of  students  under  the  two 
modes  of  administration.     There  is  now  a  link  between  the  trend  sample  and 
the  cross-sectional  sample  through  the  bridge  sample,  and  the  age-only 
subsample  of  cross -sectional  students. 
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The  following  table  shows  11  distinct  samples  of  students  used  for  the 
scaling  for  trend: 


Table  11.7 

Samples  of  Students  Used  in  Scaling  for  Trend 

Assessment  Age  9         Age  13        Age  17 

1976-77 
1981-82 
1986  Bridge  A 
1986  Bridge  B 


In  the  above,  Bridge  A  refers  to  the  tape-recorder-administered  assessment  of 
9-  and  13 -year-olds  using  the  old  age  definitions  and  time  of  assessment; 
Bridge  B  refers  to  the  tape-recorder-administered  assessment  of  9-  and  13- 
year-olds  using  the  new  age  definitions  and  time  of  assessment.     For  17 -year- 
olds,  the  Bridge  B  age  definition  and  time  of  assessment  are  identical  to 
those  used  in  past  assessments;  that  is,  for  age  17,  Bridge  A  is  identical  to 
Bridge  B. 


11.2.1    Estimation  of  Item  Parameters  and  Generation  of  Plausible  Values 

As  noted  above,  trend  data  analysis  was  carried  out  on  a  single  scale. 
The  majority  of  items  given  to  age  13  students  were  also  given  to  age  17 
students.     Therefore,  the  age  13  and  age  17  samples  were  combined  and  item 
parameters  were  estimated.     Since  there  was  only  one  item  given  in  common  to 
ages  9,  13  and  17,  and  there  were  no  items  in  common  between  ages  9  and  13  or 
ages  9  and  17,  the  item  parameters  for  the  age  9  sample  were  estimated 
separately.     The  following  table  shows  the  number  of  items  in  common  across 
the  three  ages . 


Table  11.8 

Number  of  Items  in  Uommcn  Across  the  Three  Ages 

Age  9  Aee  13         Age  17        All  Ages 


Age  9  82  0  0 

Age  13  33  42 

Age  17  35 
All  Ages  1 


The  above  table  excludes  items  that  did  not  fit  to  the  IRT  model  in  either 
cross -sectional  or  trend  data  analysis.     There  were  3,  1,  and  3  items  dropped 
from  scaling  for  ages  9,  13  and  17,  respectively.     Twenty-four  items  that 
were  given  in  1976-77  and  1981-82  but  not  in  1986  were  added  to  the  item  pool 
for  age  9.     The  fact  that  there  were  so  few  items  given  in  the  1981-82 
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X 
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special  assessment  would  have  made  the  scaling  of  the  1981-82  sample  very 
difficult  without  increasing  the  number  of  items.     Prior  to  adding  24  items, 
there  were  only  ten  science  items  common  to  the  age  9  assessments  across  the 
three  assessments.     A  list  of  the  items  scaled  for  the  three  ages,  along  with 
their  item  parameters  appears  in  Tables  E.34,  E.35,  and  E.36  in  Appendix  E. 

The  estimation  of  plausible  values  was  conducted  independently  by  age 
for  each  assessment  sample  identified  in  Table  11.7.     Because  there  were 
fewer  background  variables  available  for  trend  in  the  past,  there  were  fewer 
conditioning  variables  used  in  the  creation  of  the  plausible  values.  The 
conditioning  variables  and  the  estimated  conditioning  effects  are  given  in 
Tables  B.27  through  B.37  of  Appendix  B  for  the  three  grade/age  groups. 

11.2.2    Linking  to  the  Cross -sectional  Science  Composite 

The  units  of  the  final  trend  scale  were  determined  by  linking  each  of 
the  three  age  scales  to  the  science  composite  scale.     This  was  done  for  each 
age  by  matching  the  mean  and  standard  deviation  of  the  trend  scale  of  the 
1986  Bridge  B  sample  (the  sample  with  the  new  age  definition)  to  the  mean  and 
standard  deviation  on  the  composite  science  scale  of  the  corresponding  age 
sample.     Table  11.9  shows  the  coefficients  of  the  linear  transformations  used 
to  transform  the  trend  scale.     Note  that  the  transformation  coefficients  are 
identical  for  ages  13  and  17.     (See  Appendix  D  for  a  detailed  discussion.) 

Table  11.9 

Coefficients  of  the  Linear  Transformation 
of  the  Trend  Scale  from  the  Calibration  Units 
to  the  Units  of  the  Composite 

Age  Intercept  Slope 

9  221.73  42.43 

13  269.33  44.67 

17  269.33  44.67 


11.2.3    Extrapolation  of  the  1971-72  and  1973-74  Mean  P-Values  Results 
onto  the  Trend  Scale 

Because  of  insufficient  common  items  between  the  1971-72,  1973-74,  and 
1986  science  assessments,  data  from  1971-72  and  1973-74  were  not  included  in 
the  IRT  trend  analysis.     However,  for  the  nation  and  several  reporting 
subgroups  (e.g.;  gender)  at  each  of  the  three  age  levels,  an  estimate  of  the 
1971-72  and  1973-74  mean  level  of  student  mathematics  proficiency  was 
computed  and  is  discussed  in  this  report. 

The  method  used  to  derive  1971-72  and  1973-74  science  proficiency  scores 
is  based  on  the  strong  linear  relationship  between  the  logit  of  a  subgroup's 
weighted  mean  proportion  correct  and  its  respective  proficiency  mean  across 
the  assessments  of  1976-77,  1981-82,  and  1986,  given  an  age  level.  Assuming 
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this  linear  relationship  would  hold  for  both  1971-72  and  1973-74  data, 
extrapolation  of  proficiency  scores  of  subgroups  can  be  obtained  from' 
weighted  mean  proportion  correct  of  corresponding  subgroups  of  those  years. 
For  each  age,  separate  linear  coefficients  between  proficiency  scores  and 
difference  in  logits  of  weighted  mean  proportion  correct  were  obtained. 
Common  items  for  each  pair  of  the  three  assessment  years  1976-77,  1981-82, 
and  1986,  as  well  as  common  items  for  all  three  years,  were  used* to  calculate 
weighted  mean  proportion  correct.     These  coefficients  per  age  were  k'.;-  . 
constant  to  estimate  proficiency  scores  of  1971-72  and  1973-74  from 
differences  in  the  logits  of  the  weighted  mean  percent  correct  of  the 
corresponding  year. 

All  subgroups  in  a  given  age  were  included  in  the  regression.  The 
estimate  of  the  1973-74  proficiency  mean  for  a  subgroup  was  then  obtained  as 
the  sum  of  the  1976-77  mean  proficiency  of  the  subgroup  and  the  coefficient 
times  the  difference  between  the  logit  of  the  1973-74  and  1976-77  subgroup 
mean  proportion  correct.     Insufficient  common  items  between  1971-72  and  1976- 
77  made  it  difficult  to  extrapolate  1971-72  proficiency  scores  from  1976-77 
scores.     For  that  reason,  the  estimates  of  1971-72  proficiency  mean  were 
calculated  in  a  fashion  similar  to  that  done  for  1973-74,  except  that  1976-77 
proficiency  scores  were  replaced  by  1973-74  extrapolated  proficiency  scores. 


-255- 


CHAPTER  12 
Computer  Competence  Data  Analysis 


247 


Chapter  12 
COMPUTER  COMPETENCE  DATA  ANALYSIS 

Nancy  A.  Mead 
Educational  Testing  Service 


The  National  Assessment  of  Educational  Progress  assessed  the  computer 
competence  of  students  at  three  grade  and  age  levels  in  the  spring  of  1986 
The  levels  were  grade  3/age  9,  grade  7/age  13,  and  grade  11/age  17.     This  was 
the  first  time  that  computer  competence  had  been  assessed  by  NAEP.     The  areas 
covered  by  the  assessment  included  knowledge  of  computer  concepts, 
familiarity  with  various  computer  applications,  and  skills  in  computer 
programming. 

Initial  analysis  plans  for  computer  competence  did  not  call  for  scaling 
the  results.     Instead,  plans  were  made  to  compute  mean  percents -correct 
across  narrowly  defined  colleccions  of  items  and  to  relate  these  results  to 
background  variables.     It  was  felt  that  scaling  was  not  appropriate  for  the 
computer  competence  because  the  items  tapped  a  wide  range  of  skills  that  have 
no  underlying  pedagogical  or  psychological  continuum.     For  example,  11th 
graders  may  demonstrate  less  understanding  than  7th  graders;  withiA  a  grade 
students  may  understand  some  computer  applications  and  not  others  or  may  have 
textbook  knowledge  of  computers  and  not  user  knowledge  (or  the  converse). 

A  group  of  consultants  were  asked  to  assist  NAEP  staff  in  developing  a 
data  analysis  plan  and  in  interpreting  the  results.     Two  members  of  the  group 
were  scxected  from  the  Learning  Area  Committee  that  guided  item  development 
and  three  individuals  provided  new  perspectives.     The  group  included  a 
curriculum  developer,  a  teacher  trainer  and  researcher,  a  school  district 
computer  science  coordinator,  a  content  area  teacher  who  used  computer 
applications  in  the  classroom,  and  a  writer  who  specialized  in  computer- 
related  topics. 

The  first  meeting  was  devoted  to  determining  the  specifications  for  the 
data  analysis.     The  analysis  group  reviewed  the  items  in  tie  pool,  their 
classification  according  to  the  computer  competence  objectives,  and  their 
unweighted  percents-correct .     Many  items  were  very  difficult  for  students. 
Often  the  percentage  of  correct  responses  fell  close  to  the  chance  level.  As 
an  initial  analysis  step,  the  group  recommended  that  item  analyses  be 
conducted  and  that  items  that  did  not  discriminate  between  high  and  low 
achievers  be  eliminated.     The  group  also  recommended  that  two  items  be 
eliminated  from  the  analysis.     One  of  these  questions  did  not  have  a  clear 
correct  answer  and  the  other  had  a  printing  error. 

The  analysis  group  reviewed  the  knowledge  items  and  proposed  that  they 
be  reported  by  two  categories:  textbook  knowledge  and  practical  knowledge. 
The  first  category  deals  with  concepts  usually  presented  in  textbooks,  for 
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example,  the  history  of  computers  or  terminology.     The  second  category  deals 
with  knowledge  that  could  be  gained  through  use  of  a  computer. 

The  analysis  group  classified  the  application  items  into  five 
categories:     distinguishing  between  applications,  word  processing,  databases, 
graphics,  and  spreadsheets.    The  first  category  of  items  asked  simple 
questions  about  what  application  programs  should  be  used  to  accomplish 
various  tasks.    The  other  categories  focused  on  specific  applications. 
Later,  when  the  group  reviewed  the  results,  they  decided  that  the  first 
category  did  not  warrant  separate  reporting  and  these  items  were  reassigned 
to  the  appropriate  application  categories.     The  computer  competence 
objectives  encompassed  several  additional  applications:  laboratory 
instrumentation,  telecommunication,  music  generation,  and  models  and 
simulations.     However,  the  analysis  group  felt  that  there  were  not  enough 
items  in  these  areas  to  warrant  reporting. 

The  analysis  group  decided  that  the  programming  items  should  be  reported 
by  programm5.ng  language:     BASIC,  Logo,  and  Pascal.     Within  Pascal,  two  types 
of  items  wer^  identified:     generic  and  specific.     Students  might  be  able  to 
answer  items  in  the  first  category  if  they  had  a  knowledge  of  a  programming 
language  other  than  Pascal.     The  second  category  of  items  required  knowledge 
of  language  structures  specific  to  Pascal. 

The  analysis  group  also  identified  student  background  variables  that 
should  be  used  in  the  analyses.     These  included  general  demographic 
characteristics  as  well  as  factors  that  were  specific  to  students'  experience 
with  computers.     A  few  additional  variables  were  added  by  NAEP  staff.  The 
final  list  of  general  variables  included: 

student  sex 

student  race  or  ethnic  background 
student  age 

parents'  level  of  education 

region  of  the  country 

size  and  type  of  community 

type  of  school  (public  or  private) 

student's  high  school  program  (general,  academic  or 
vocational- -grade  11/age  17  only) 

The  computer  related  variables  included: 

Has  the  student  ever  used  a  computer? 

Does  the  student's  family  own  a  computer? 

Is  the  student  studying  computers  in  school  now?  (grade 

3/age  9  and  grade  7/age  13  only) 

Is  the  student  using  computers  to  practice  math,  reading 
or  spelling?  (grade3/age  9  and  grade  7/age  13  only) 
Is  the  student  currently  taking  a  class  in  computers? 
(grade  11/age' 17  only) 
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Has  the  student  taken  a  computer  literacy  class'  (erade 
11/age  17  only) 

Has  the  student  taken  a  computer  programming  class? 
(grade  11/age  17  only) 

The  group  also  wanted  to  look  at  the  programming  language  means  in  relation 
l?n,,t  ""^^^^^^^^  programming  languages  does  the  student  know?"  and 

Which  programming  language  does  the  student  know  best?" 

In  addition  to  the  variables  listed  above,  the  analysis  group  wanted  to 
develop  a  composite  variable  that  indicated  various  levels  of  exposure  to 
computers.     After  considering  several  approaches,   the  group  settled  on  a 
variable  that  crossed  "Does  the  student's  family  own  a  computer?"  and  "Is  the 
student  studying  computers  in  school  now?"  (or,  at  grade  11/age  17     "Is  the 
student  currently  taking  a  class  in  computers?").     Four  different  categories 
of  exposure  were  created: 


1)      student's  family  has  a  computer  and  student  is  studying 
computers ; 

student's  family  has  a  computer  and  student  is  not 
studying  computers; 

student's  family  doesn't  have  a  computer  and  student  is 
studying  computers;  and 

student's  family  doesn't  have  a  computer  and  student  is 
not  studying  computers. 


2) 
3) 
4) 


As  anticipated,  the  highest  and  lowest  levels  of  exposure  were  associated 
with  the  highest  and  lowest  levels  of  computer  competence. 

^..^.^^f""  ^^'^^  conducted  for  all  items.     Even  though  many  items  were 

ditticult,  all  Items  discriminated  between  high  and  low  achievers. 

Analyses  were  then  conducted  for  all  students  in  grades  3,  7,  and  11  and 
for  all  subgroups  defined  by  the  background  variables.     Weighted  percents- 
correct  were  computed  for  each  item.     Finally,  weighted  mean  percents-correct 
were  computed  across  all  items  and  across  all  categories  of  items  identified 
by  the  analysis  group. 

Percents-correct  and  mean  percents-correct  were  computed  according  to 
the  formula:  ° 


Rights 


Rights  +  Wrongs  +  Omits 


Students  who  did  not  reach  an  item  were  excluded  from  the  analyses  The 
percentages  of  students  who  did  not  reach  an  item  was  no  greater  than  5 
percent  except  for  some  items  in  the  BASIC  and  Logo  sections  for  the  3rd 
graders  and  for  some  items  in  the  Logo  section  for  the  7th  graders  Means 
that  included  any  items  with  a  not  reached  greater  than  5  percent  were 


flagged  with  a  warning:  "Less  than  95  percent  of  the  population  reflected  In 
the  mean." 

A  small  number  of  the  computer  competence  items  were  open-ended. 
Students  provided  responses  In  the  form  of  lines  of  computer  code  or  brief 
written  explanations.    These  responses  were  rated  by  professional  scorers. 
To  document  the  consistency  of  scorers,  a  20  percent  sample  of  the  items  were 
independently  rated  by  a  second  scorer,     (See  Chapter  6.2  for  a  description 
of  professional  scoring  and  scorer  reliability.) 

At  their  second  meeting  the  analysis  group  reviewed  the  results  and 
offered  their  interpretations  of  findings.     The  observations  of  the  group 
were  incorporated  into  the  assessment  report. 
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Chapter  13 
HISTORY  AND  LITERATURE  DATA  ANALYSIS 


Rebecca  Zwick 
Educational  Testing  Service 


History  and  literature  items  were  included  in  four  of  the  92  booklets 
administered  to  9,774  students  who  were  17  years  old  or  in  grade  11  in  the 
1986  assessment.     These  items  were  administered  for  the  first  time  in  the 
1986  assessment.     Each  of  i-he  four  booklets  contained  one  of  four  history 
blocks  (HI,  H2,  H3,  or  H4) ,  one  of  four  literature  blocks  (LI,  L2,  L3,  or 
L4) ,  and  reading  block  R4.     (That  is,  the  history  and  literature  blocks  were 
not  BIB-spiralled. )     The  history  blocks  consisted  of  34  to  36  cognitive  items 
and  a  common  set  of  25  history  background  and  attitude  items;  the  literature 
blocks  contained  30  to  31  cognitive  items,  as  well  as  42  literature 
background  and  attitude  items.     All  history  and  literature  items  were 
multiple  choice. 


13.1    ITEM  ANALYSIS  AND  DIMENSIONALITY  ASSESSMENT 

Within  each  of  the  two  subject  areas,  the  four  blocks  were  constructed 
to  be  similar  in  content  and  difficulty.     Table  13.1  shows  the  number  of 
items,  KR-20  reliability,  average  tetrachoric,  mean,  standard  deviation,  and 
mean  percent  correct  for  each  block.     Within  each  content  area,  the  item 
analysis  results  are  quite  similar  across  blocks,  although  the  history  blocks 
differed  somewhat  in  difficulty. 

A  series  of  analyses  of  differential  item  functioning  (see  Holland  & 
Thayer,  1986)  on  the  history  and  literature  items  was  also  performed.  (The 
analyses  of  history  items  are  detailed  in  Zwick  &  Ercikan,   in  press.)  The 
purpose  of  these  analyses  was  to  identify  items  on  which  there  were 
performance  differences  across  racial/ethnic  and  gender  groups  who  were 
matched  on  overall  score  (and  in  some  cases,  on  exposure  to  relevant 
instruction).     After  examining  the  results  of  the  analyses,   it  was  determined 
that  removal  of  the  items  that  showed  differential  functioning  would 
seriously  impair  the  validity  of  the  test.     For  example,  all  three  history 
items  showing  substantial  differential  performance  in  the  White  student- Black 
student  comparison  were  items  on  Black  history,  on  which  Black  students 
performed  better  than  a  matched  group  of  White  students.     Removal  of  Black 
history  items,  however,  would  clearly  be  undesirable. 


Laurel  Barnett  and  David  Freund  provided  statistical  programming. 
Robert  Mislevy  and  Kathleen  Sheehan  provided  consultation  on  scaling.  David 
Freund  prepared  the  documentation  of  history  and  literature  derived  variables 
contained  in  Table  F.l  of  Appendix  F. 
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Table  13.1 


NAEP  History  and  Literature  Assessment 
Descriptive  Statistics 


Block 

Nvimbers  of 
Items 

KR-20 
Reliability 

.84 

Average 
Tetrachoric 

Mean  Number 
Correct 

S. 

D. 

Mean  Percer 
Correct 

HI 

36 

.39 

20.8 

6. 

3 

.58 

H2 

36 

.83 

.35 

19.2 

6. 

4 

.53 

H3 

35 

.82 

.40 

16.9 

6 

1 

.48 

H4 

34 

.87 

.48 

19.2 

6 

9 

.57 

LI 

30 

.78 

.35 

15.1 

5 

.2 

.51 

L2 

31 

.76 

.30 

15.6 

4 

.9 

.50 

L3 

30 

.77 

.30 

16.4 

4 

.9 

.55 

L4 

30 

.72 

.24 

15.7 

4 

.5 

.52 

Sample  size  for  each  analysis  is  approximately  1950, 
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For  purposes  of  dimensionality  assessment  in  NAEP,  we  have  investigated 
the  full -information  factor  analysis  method  developed  by  Bock,  Gibbons,  and 
Muraki  (1985;  see  Zwick,  1987a),  which  provides  an  elegant  model  for 
assessing  the  dimensionality  of  dichotomous  item  responses.  Unfortunately, 
the  implementation  of  the  full- information  solution  in  the  TESTFACT  program 
(Wilson,  Wood,  &  Gibbons,  1983)  is  expensive  and  sometimes  results  in 
convergence  problems.     Furthermore,  there  is  some  evidence  that  heavy 
reliance  on  the  significance  tests  provided  can  lead  to  over- factoring.  At 
present,  therefore,  we  have  chosen  to  use  a  less  elegant,  but  apparently 
satisfactory  procedure:     Through  a  nonstandard  application  of  the  TESTFACT 
program,   it  is  possible  to  perform  a  MINRES  (Harman,   1976)  factor  analysis  of 
a  matrix  of  tetrachoric  correlations  that  have  been  corrected  for  guessing 
using  a  modification  of  Carroll's  (1945)  correction. 

The  results  of  an  analysis  of  this  kind  are  shown  in  Table  13.2.  In 
each  of  the  eight  blocks,  a  large  first  factor  was  in  evidence;  the  size  of 
the  second  factor  ranged  from  6  to  13  percent  of  the  first  in  history  and 
from  16  to  21  percent  of  the  first  in  literature.     Careful  examination  of  the 
items  loading  on  the  second  factors,  both  before  and  after  rotation,  revealed 
no  content-based  interpretation  for  these  factors.  For  comparison,  results  of 
a  similar  analysis  of  NAEP  reading  items  is  shown.   (This  analysis,  conducted 
in  1984,   is  not  strictly  parallel  with  the  present  analyses  in  that  42  items 
intended  to  represent  four  distinct  types  of  reading  items  were  selected  from 
a  larger  pool  for  inclusion.)    As  in  the  case  of  the  reading  items,  the 
decision  was  made  that  the  history  and  literature  data  could  be  well 
summarized  by  a  single  history  scale  and  a  single  literature  scale. 


13.2  SCALING 

As  in  the  case  of  reading,  mathematics,  and  science,   item  response 
theory  (IRT)  methods  were  used  to  derive  a  history  scale  and  a  literature 
scale  based  on  the  cognitive  items.     Scale  values  were  obtained  for  only  the 
7,812  students  who  were  in  grade  11;  scale  values  were  not  obtained  for  17- 
year-olds  who  were  in  grades  other  than  11. 

The  scaling  of  history  and  literature  differed  in  two  ways  from  the  IRT 
scaling  approach  used  in  other  NAEP  subject  areas.     First,  because  the 
history  and  literature  assessment  consisted  of  four  nonoverlapping  sets  of 
items  in  each  of  the  two  subject  areas,   it  was  necessary  to  make  the 
assumption  that  the  four  blocks  of  items  within  a  subject  area  were 
equivalent  samples  of  the  content  domain.     This  assumption  is  reasonable 
since  the  blocks  were  constructed  to  be  similar  in  content,  and  was  supported 
by  the  item  analysis  results  in  Table  13.1.     The  second  difference  is  that, 
because  of  the  relatively  large  number  of  items  administered  to  each  student, 
background  data  were  not  used  to  improve  the  estimation  of  students' 
proficiency  distributions.     That  is,  there  were  no  conditioning  variables 
(see  Chapter  8) , 

Scaling  was  achieved  through  a  straightforward  application  of  the  BILOG 
program  (Mislevy  &  Bock,  1982).     Within  each  content  area,  a  posterior 
distribution  was  estimated  for  eac'i  student,  based  on  that  student's  observed 


Table  13.2 
Results  of  MINRES  Factor  Analyses  of 
Guessing-Corrected  Tetrachoric  Matrices  of  NAEP  Items 


Number  of  Items 


Percent  of  Explained  Variance 
First  Factor         Second  Factor 


1984  Reading 
(Grade  8/Age  13) 


42 


29 


NAEP  1986  History  and  Literature  (Grade  11) 


1986  History 
(Grade  11) 


1986  Literature 
(Grade  11) 


Block 

HI 

36 

40 

5 

H2 

36 

38 

5 

H3 

35 

42 

5 

H4 

34 

51 

3 

Block 

LI 

30 

38 

6 

L2 

31 

36 

7 

L3 

30 

33 

7 

L4 

30 

38 

6 

^^Sample  size  for  each  analysis  is  approximately  1950. 
Items  were  chosen  to  represent  four  distinct  item  types. 
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item  responses  and  on  the  estimated  grade  11  population  distribution.  The 
mean  of  the  posterior  distribution  was  then  taken  as  the  proficiency  estimate 
for  that  student  in  the  provisional  6  metric.     That  is, 

-  E(^?|x^)  -  J^p(^?|x^.a,b,c)d^? 

-  |^p(x^|^?,a,b,c)g(^?)d^?  /  |p(Xj^|^?,a,b,c)g(^?)d^?, 


where 


is  the  proficiency  estimate  for  the  i  examinee, 

Xji^  is  the  vector  of  item  responses  for  the  i'  examinee, 

a,  b,  and  c    are  the  vectors  of  item  parameters  for  the  three - 
parameter  logistic  model,  and 

g(^)  is  the  (estimated)  distribution  of  proficiency  in  the 

nation. 


This  distribution  was  approximated  as  a  histogram  over  20  equally  spaced 
points  from  -5  to  +5  in  the  manner  described  in  Mislevy  (1984),  For 
reporting  purposes,  the  estimated  proficiencies  were  transformed  so  that,  for 
both  history  and  literature,  the  grade  11  mean  and  standard  deviation  were 
285  and  40,  respectively. 

Item  parameter  estimates  for  the  history  and  literature  items  are  given 
in  Appendix  E  in  the  original  BILOG  calibration  metric.     In  order  to  obtain 
results  in  terms  of  the  proficiency  scales  used  for  reporting  (i,e,,  means  of 
285,  standard  deviations  of  40),  the  scale  values  that  would  be  obtained 
using  the  parameter  estimates  in  Appendix  E  need  to  be  transformed.  The 
relation  between  the  provisional  scales  from  BILOG  (^H  and        for  history  and 
literature,  respectively)  and  the  final  reporting  scales  (RSH  and  RSL)  are 

RSH  -  43.60  (^H)  +  285.08  and 
RSL  -  45.36  (^L)  +  285.02. 

The  corresponding  changes  to  the  item  parameter  estimates  are 
^RSH  -  "<?h/^3-^0'  ^RSH  -  ^^-60  285.08  and  a^^^  - 

^RSL  ^  ^^*^^^^L  ^  285.02.     The  c  parameters  are  the  same  in  both  metrics 
^RSH  ^  ""en         ^RSL  ^  ^^L>- 


13.3    ANALYSES  USING  BACKGROUND  VARIABLES 

The  NAEP  report  Literature  and  U.S.  History  (Applebee,  Langer,  &  Mullis, 
1987)  includes  analyses  based  on  variables  derived  from  the  responses  to  the 
background  and  attitude  questions  contained  in  the  history  and  literature 
blocks.     The  definitions  of  these  variables  are  given  in  Table  F.l  of 
Appendix  F. 


-269- 


13.4    ANALYSES  USING  MEAN  PERCENTS  -  CORRECT 

The  Literature  and  U.S.  History  report  also  contained  analysis  based  on 
mean  percent  correct  for  sets  of  items  in  certain  categories.     The  items 
included  in  each  of  these  analyses  are  listed  in  Table  F.2  of  Appendix  F. 
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Chapter  14 

WEIGHTING  PROCEDURES  AND  VARIANCE  ESTIMATION*" 


Eugene  G.  Johnson 
Educational  Testing  Service 

John  Burke,  Jill  Braden,  Morris  H.  Hansen, 
Josef ina  A.  Lago,  and  Benjamin  J.  Tepping 

Westat,  Inc. 


As  was  the  case  in  previous  assessments,  the  1986  National  Assessment 
used  a  complex  sample  design  with  the  goal  to  obtain  a  sample  from  which 
estimates  of  population  and  subpopulation  characteristics  could  be  obtained 
with  reasonably  high  precision  (as  measured  by  low  sampling  variability) .  At 
the  same  time,   it  was  necessary  that  the  sample  be  economically  and 
operationally  feasible  to  obtain.     The  resulting  sample  had  certain 
properties  that  had  to  be  taken  into  account  in  the  proper  analysis  of  the 
data  from  the  assessment. 

The  1986  NAEP  sample  was  obtained  through  a  stratified  multistage 
probability  sampling  design  that  included  provisions  for  sampling  certain 
subpopulations  at  higher  rates  (see  Chapter  3).     To  account  for  the 
differential  probabilities  of  selection,  each  student  was  assigned  a  sampling 
weight.     Section  14.1,  below,  will  discuss  the  procedures  used  to  derive 
these  sampling  weights. 

Another  consequence  of  the  NAEP  sample  design  is  its  effect  on  the 
estimation  of  sampling  variability.     Because  of  the  effects  of  cluster 
selection  (students  within  schools,  schools  within  primary  sampling  units) 
and  because  of  the  effects  of  certain  adjustments  to  the  sampling  weights 
(nonresponse  and  poststratif ication) ,  observations  made  on  different  students 
cannot  be  assumed  to  be  independent  of  one  another.     As  a  result,  ordinary 
formulae  for  the  estimation  of  the  variance  of  sample  statistics,  based  on 
assumptions  of  independence,  will  tend  to  underestimate  the  true  sampling 
variability.     Section  14.2  will  discuss  the  jackknifing  technique  used  by 
NAEP  to  estimate  sampling  variability. 

Since  the  sample  design  determines  the  derivation  of  the  sampling 
v/eights  and  the  estimation  of  sampling  -  friability,   it  will  be  helpful  to 
note  the  key  features  of  the  1986  NAEP  sample  design.     More  detailed 
descriptions  of  the  design  appear  in  Chapter  3  of  this  report  and  in  National 
Assessment  of  Educational  Progress -- 17th  Year  Sampling  and  Weighting 


The  tables  of  design  effects  were  produced  by  David  Freund. 
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Procedures  Final  Report  (Burke,  Braden,  Hansen,  Lago,  &  Tepping,  1987), 
the  final  report  prepared  by  Westat,  Inc.,  the  firm  subcontracted  by  ETS  to 
select  the  sample. 

The  target  population  in  1986  consisted  of  9-year-olds,  13-year-olds, 
and  17 -year- olds  enrolled  in  public  and  private  elementary  and  secondary 
schools,  along  with  other  students  in  the  modal  grade  for  each  of  these  three 
ages  as  these  ages  were  defined.     The  1986  sample  was  a  multistage 
probability  sample  consisting  of  four  stages  of  selection.     The  first  stage 
of  selection,  the  primary  sampling  units  (PSUs) ,  consisted  of  counties  or 
groups  of  counties.     The  second  stage  of  selection  consisted  of  elementary 
and  secondary  schools.     The  assignment  of  sessions  to  sampled  schools 
comprised  the  third  stage  of  sampling,  and  the  fourth  stage  involved  the 
selection  of  students  within  schools  and  their  assignment  to  sessions.  The 
probabilities  of  selection  of  the  first-  and  second-stage  sampling  units  were 
proportional  to  measures  of  their  size,  while  the  probability  for  subsequent 
stages  of  selection  were  such  that  the  overall  probabilities  of  selection  of 
students  were  approximately  uniform,  with  exceptions  for  certain 
subpopulations  that  were  oversampled  by  design.     Students  from  schools  with 
relatively  high  concentrations  of  Black,  Hispanic,  or  Asian  American  students 
were  deliberately  sampled  at  twice  the  normal  rate  to  obtain  larger  samples 
of  respondents  from  those  subpopulations  in  order  to  increase  the  precision 
in  the  estimation  of  the  characteristics  of  these  subpopulations.  Students 
from  schools  with  smaller  numbers  of  eligibles  received  lower  probabilities 
of  selection. 

A  major  change  in  the  1986  assessment  from  prior  assessments  was  a  shift 
to  spring  assessment  for  each  age  class,  coupled  with  a  change  in  the  age 
definitions  for  the  9-  and  13-year-old  students.     In  previous  assessments, 
13-year-olds  were  defined  on  a  calendar-year  basis  and  were  assessed  in  the 
fall;  9-year-olds  were  also  defined  on  a  calendar-year  basis  and  were 
assessed  in  the  winter;  and  17-year-olds  were  defined  on  an  October- through- 
September  basis  and  assessed  in  the  spring.     For  the  spiral  (main)  assessment 
in  1986,  the  students  of  each  of  the  three  age  classes  were  defined  as  the 
students  born  between  October  1  and  September  30  of  the  appropriate  years. 
All  ages  were  assessed  in  the  spring  for  the  spiral  assessment  of  1986.  To 
determine  the  possible  effects  of  changes  in  age  definitions  and  time  of 
assessment  as  well  as  the  effects  of  change  in  mode  of  administration 
(elimination  of  the  audiotape  used  for  pacing  the  exercises),  two  bridge 
studies  were  also  conducted. 

The  full  1986  NAEP  assessment  thus  includes  a  niomber  of  different 
samples  from  several  populations.     Each  of  these  samples  has  its  own  set  of 
weights  that  are  to  be  used  to  produce  estimates  about  the  characteristics  of 
the  population  addressed  by  the  sample  (the  target  population).     The  various 
samples  and  their  target  populations  are  as  follows: 

The  Spiral  Samples  of  Students.     These  samples,  one  for  each  of  the 
three  grade/age  combinations,  were  drawn  in  the  spring,  use  the  new  age 
definitions,  and  consist  of  all  students  assessed  in  the  main  (spiral) 
assessment.     The  target  population  for  each  of  these  samples  consists  of  all 
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students  who  are  in  the  specified  grade/age  combination  who  were  deemed 
assessable  by  their  school. 

The  Bridge  A  Samples  of  Students.     For  each  of  the  ages  9  and  13,  there 
are  three  bridge  samples  (from  booklets  1,  2  and  3)  designed  to  allow  the 
measurement  of  the  effect  of  changing  the  age  definitions  and  the  time  of 
year  the  assessment  data  were  collected.   (Since  these  changes  affected  only 
ages  9  and  13,  a  bridge  sample  was  not  necessary  for  age  17.     Since  trend 
data  have  been  traditionally  collected  only  by  age,  grade  sampling  was 
unnecessary.)     These  samples  were  drawn  at  the  old  times  of  assessment  and 
use  the  old  age  definitions.     The  target  population  for  each  of  these  samples 
consists  of  the  assessable  age  eligibles,  using  the  old  age  definitions. 

The  Bridge  B  Samples  of  Students.     For  each  of  the  three  ages  (9,  13, 
and  17)  there  are  two  bridge  samples  (from  booklets  4  and  5)  designed  to 
allow  the  measurement  of  the  effect  of  changing  from  tape-recorded  to  printed 
mode  of  administration  for  mathematics  and  science.     These  samples  were  drawn 
in  the  spring  and  use  the  new  age  definitions.     The  target  population  for 
each  of  these  samples  consists  of  all  students  who  are  of  the  specified  age, 
using  the  new  age  definitions. 


14.1    DERIVATION  OF  THE  SAMPLE  WEIGHTS 

As  indicated  previously,  NAEP  uses  differential  sampling  rates, 
deliberately  oversampling  certain  subpopulations  to  obtain  larger  samples  of 
respondents  from  those  subgroups  thereby  enhancing  the  precision  of  estimates 
of  characteristics  of  these  oversampled  subgroups.     As  a  result  of  this 
oversampling,  these  subpopulations,  corresponding  to  students  from  schools 
with  high  concentrations  of  Black  or  Hispanic  students,  are  overrepresented 
in  the  sample.    Appropriate  estimation  of  population  characteristics  must 
take  this  disproportionate  representation  into  account.     This  is  accomplished 
by  assigning  a  weight  to  each  respondent,  where  the  weights  properly  account 
for  the  sample  design  and  reflect  the  appropriate  proportional  representation 
of  the  various  types  of  individuals  in  the  population. 

The  weighting  procedures  for  1986  included  computing  the  student's  base 
weight,  the  reciprocal  of  the  probability  that  the  student  was  invited  to  a 
particular  session.     These  base  weights  were  then  adjusted  for  nonresponse 
and  then  subjected  to  a  trimming  algorithm  to  reduce  excessively  large 
weights.     The  weights  were  further  adjusted  by  a  poststratif ication  procedure 
in  an  effort  to  reduce  the  sampling  error  and  certain  potential  biases  of 
estimates  relating  to  student  populations  corresponding  to  several  subgroups 
of  the  total  population.     Poststratif ication  was  performed  by  adjusting  the 
weights  of  the  sampled  students  so  that  the  resulting  estimates  of  the  total 
number  of  students  in  a  number  of  specified  subgroups  of  the  population 
corresponded  to  population  totals  based  on  information  from  the  Current 
Population  Survey,  the  1980  Census,  and  from  NAEP.     The  subpopulations  were 
defined  in  terms  of  race,  ethnicity.  Office  of  Business  Economics  region,  and 
sampling  descriptor  of  community  (based  on  the  size  and  degree  of 
urbanization  of  a  county) . 
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The  following  sections  provides  an  overview  of  the  procedures  used  to 
derive  the  sampling  weights.     Further  details  In  the  derivation  of  these 
weights  can  be  found  in  Burke  *-*t  al.  (1987). 


14.1.1    Student  Base  Weight 

TVie  base  weight  assigned  to  a  student  is  the  reciprocal  of  the 
probability  that  the  student  was  invited  to  a  particular  type  of  assessment 
session,  that  Is,  a  spiral  session  or  a  particular  bridge  assessment  session. 
That  probability  is  the  product  of  four  factors 

1)  the  probability  that  the  PSU  was  selected; 

2)  the  conditional  probability,  given  the  PSU,  that  the  school 
was  selected; 

3)  the  conditional  probability,  given  the  sample  of  schools 
in  a  PSU,  that  the  school  was  allocated  the  specified 
type  of  session;  and 

4)  the  conditional  probability,  given  the  school,  that  the 
student  was  invited  to  the  specified  type  of  session 

Thus,  the  base  weight^  for  a  student  may  be  expressed  as  the  product 

«  PSUWT  *  SCHWT  *  SESSWT  *  STUDWT 

where  PSUWT,  SCHWT,  SESSWT,  and  STUDWT  are,  respectively,  the  reciprocals  of 
the  preceding  probabilities. 


14.1.2        Adjustment  of  Base  Weights  for  Nonresponse 

The  base  weight  for  a  student  was  adjusted  by  three  nonresponse  factors 
one  to  adjust  for  noncooperating  schools,  the  second  to  adjust  for  allocated 
sessions  that  were  not  conducted,  and  the  third  to  adjust  for  students  who 
were  invited  to  the  assessment  but  did  not  appear  either  in  the  scheduled 
session  or  in  a  makeup  session.     Thus,  the  student  nonresponse  adjusted 
weight  is  of  the  form 

y  *f    *f  *f 

w      ^B      ^1      ^2  ^3 


where  W^  is  the  student  base  weight,  f^  is  a  school  nonresponse  factor,  f^  i 
a  session  nonresponse  factor,  and  f^  is  a  student  nonresponse  factor,  eacn 
computed  as  described  below. 


14.1.2.1  School  Nonresponse  Adjustment 

School  nonresponse  adjustments  were  intended  to  compensate  for  school 
nonresponse  occurring  prior  to  spiral  and  tape  session  assignment.  These 
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factors  were  computed  separately  within  each  PSU  for  up  to  three  classes  of 
schools  using  as  many  nonresponse  classes  as  the  number  of  sampled  schools  in 
the  PSU  and  nonresponse  pattern  allowed.     In  most  cases,  only  one  class  was 
identified  in  the  PSU  for  each  of  the  three  age  groups. 

For  any  nonresponse  class,  c,  the  school  nonresponse  factor  for  schools 
selected  for  spring  assessment  (either  spiral  or  Bridge  B)  is  given  by 

S  W.G. 

f.     -      i.A      "  ^ 


'Ic 


2  W.G. 

i.B      ^  ^ 


where 

-  school  weight  (the  reciprocal  of  the  probability  of 

selection  of  the  school  conditional  on  the  PSU) ; 

G.  «  estimated  number  of  grade  and  age  eligible  students 

in  school  i; 

set  A         consists  of  the  original  sample  of  eligible  schools 
(including  refusing  schools  but  not  including 
substitutes);  and 

set  B  consists  of  all  cooperating  schools  at  the  time  of 

session  allocation  (including  schools  that  were 
substituted  for  noncooperating  schools). 

Note  that,  for  a  substitute  school,  W.   (SCHWT)  was  defined  as  the  school 
weight  of  the  originally  selected  school  for  which  it  was  a  substitute.  The 
sampling  rate  of  students  within  a  substititute  school  (and  hence  the  value 
of  STUDWT  for  such  a  school)  was  defined  using  enrollment  data  for  the 
substitute  itself. 

A  similar  formula  was  used  for  the  school  level  nonresponse  adjustment 
for  the  Bridge  A  schools  with  the  exception  that,  since  only  age-eligible 
students  were  selected  in  the  Bridge  A  samples,   the  factor  O     is  replaced  by 
A^,   the  number  of  age -eligible  students  in  the  school. 


14.1.2.2      Session  Nonresponse  Adjustment 

The  session  nonresponse  adjustments  were  intended  to  compensate  for 
school  nonresponse  occurring  after  spiral  and/or  tape  session  assignment  in 
the  spring- selected  schools.     (No  session  nonresponse  adjustment  was  needed 
for  Bridge  A  schools  since  those  schools  were  allocated  tape  sessions  only. 
For  those  schools,  session  nonresponse  is  equivalent  to  school  nonresponse 
and  is  adjusted  for  as  in  section  14.1.2.1.) 
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For  spiral  sessions,  the  adjustment  was  computed  separately  by  PSU  for 
one  or  two  classes  of  schools  in  each  of  the  three  grade/age  groups  and  is  a 
ratio  of  the  form 


2c 


S 

i£A 


W*.G. 


1  1 


2 

i£B 


W*.G. 


1  1 


where        is  the  number  of  grade/age  eligibles  as  before,  W*,^^  is  the  product 
of  the  school-nonresponse  adjusted  school  weight  and  the  session  weight  (i.e. 
f-     *  SCHWT  *  SESSWT  ),  the  set  A  consists  of  all  spiral-allocated  schools  in 
tne  nonresponse  class  c  within  the  PSU  who  were  cooperating  at  the  time  of 
spiral  session  allocation,  and  the  set  B  consists  of  all  such  schools  who 
ultimately  cooperated. 

For  the  Bridge  B  tape  sessions,  the  adjustment  was  computed  separately 
in  each  PSU  for  each  of  the  two  tape  booklets  in  each  of  the  three  age  groups 
and  was  of  similar  form  to  f^^,  above,  with  two  exceptions.     First,  the 
factor  G.  was  replaced  by  A.,  the  number  of  age  eligibles  in  the  school. 
Second,  hhe  sets  A  and  B  were  restricted  to  those  schools  in  the  PSU  who  were 
allocated  the  particular  tape  booklet. 

14 . 1 . 2 . 3    Student  Nonresponse  Adjustment 

The  student  nonresponse  adjustment  was  computed  separately  within  a  PSU 
by  age  group  for  each  of  the  two  tape  booklets  in  Bridge  B  sessions  and  for 
each  of  the  three  tape  booklets  in  Bridge  A  sessions.      For  tape  booklet  t  in 
PSU  h,  the  student  nonresponse  adjustment  f^  was  computed  as  the  ratio  of  the 
sum  of  the  weights  of  all  students  in  the  PSU  who  were  invited  to  a  tape 
booklet  t  session  divided  by  the  sum  of  the  weights  of  all  students  in  the 
PSU  who  actually  participated  in  a  tape  booklet  t  session;  where  the  student 
weights  were  adjusted  for  school-  and  session-nonresponse. 

For  spiral  sessions,  the  student  nonresponse  adjustment  was  made 
separately  within  each  PSU  for  two  sets  of  students:     those  in  or  above  the 
grade  modal  for  their  age  and  those  in  a  grade  below  that  modal  for  their 
age,  and  is  the  ratio  of  the  sum  of  the  (school-  and  session-nonresponse 
adjusted)  weights  of  all  students  in  the  given  age  set  who  were  invited  to  a 
spiral  session  divided  by  the  sum  of  the  adjusted  weights  of  all  such 
students  who  actually  participated.     The  use  of  two  age  sets  for  nonresponse 
adjustment  is  in  recognition  of  I:he  likely  differences  between  suudents  in 
the  two  sets  both  in  their  assessed  abilities  and  in  their  likelihood  of 
nonresponse . 

14.1.3        Trimming  of  Weights 

In  a  number  of  cases,  students  were  assigned  extremely  large  weights. 
One  cause  of  large  weights  was  underestimation  of  the  number  of  eligible 
students  in  some  schools  leading  to  inappropriately  low  probabilities  of 
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selection  for  those  schools.     Other  extremely  large  weights  arose  as  the 
result  of  high  levels  of  nonresponse  coupled  with  low  to  moderate 
probabilities  of  selection. 

Students  with  extremely  large  weights  have  an  unusually  large  impact  on 
estimates  such  as  weighted  means.     Since  the  variability  in  weights 
contributes  tg  the  variance  of  an  overall  estimate  by  an  approximate  factor  1 
+  V  ,  where  V    is  the  relative  variance  of  the  weights,  ^  few  extremely  large 
weights  are  likely  to  produce  large  sampling  variances  of  the  statistics  of 
interest,  especially  when  the  large  weights  are  associated  with  students  with 
atypical  performance  characteristics. 

To  reduce  this  problem,  a  procedure  of  trimming  the  more  extreme  weights 
to  values  somewhat  closer  to  the  mean  weight  was  applied.     This  trimming  can 
increase  t^e  accuracy  of  the  resulting  survey  estimates,  substantially 
reducing  V    and  hence  the  sampling  variance  while  introducing  a  small  bias. 
The  trimming  algorithm  was  identical  to  that  used  in  the  1984  NAEP  and  had 
the  effect,  approximately,  of  trimming  the  weight  of  any  school  that 
contributed  more  than  a  specified  proportion,  f ,  to  the  estimated  variance  of 
the  estimated  number  of  students  eligible  for  assessment.     The  trimming  was 
done  separately  for  the  spiral  assessment  and  for  each  tape  booklet  in  each 
of  the  bridge  samples.     In  each  case,  the  value  of  the  proportion  f  was 
chosen  to  be  10/M,  where  M  was  the  number  of  schools  in  which  a  specified 
assessment  was  conducted.     The  number  of  schools  whose  weights  were  trimmed 
was  small,  being  between  0  and  5  in  each  of  the  samples. 

14 . 1 . 4       Poststratif icat ion 

As  in  most  sample  surveys,  the  respondent  weights  are  random  variables 
that  are  subject  to  sampling  variability.     Even  if  there  were  no  nonresponse, 
the  respondent  weights  would  at  best  provide  unbiased  estimates  of  the 
various  subgroup  proportions.     However,  since  unbiasedness  refers  to  average 
performance  over  a  conceptually  infinite  number  of  replications  of  the 
sampling,  it  is  unlikely  that  any  given  estimate,  based  on  the  achieved 
sample,  will  exactly  equal  the  population  value.     Furthermore,  the  respondent 
weights  have  been  adjusted  for  nonresponse  and  a  number  of  extreme  weights 
have  been  reduced  in  size. 

To  reduce  the  mean  squared  error  of  estimates  using  the  sampling 
weights,  these  weights  were  further  adjusted  so  that  estimated  population 
totals  for  a  niomber  of  specified  subgroups  of  the  population,  based  on  the 
sum  of  weights  of  students  of  the  specified  type,  were  the  same  as  presumably 
better  estimates  derived  from  other  sources.     This  adjustment,  called 
poststratification,  especially  reduces  the  mean  squared  error  of  estimates 
relating  to  student  populations  that  span  several  subgroups  of  the 
population.     The  poststratification  was  done  separately  for  the  spiral 
sessions  and  each  of  the  bridge  sessions  within  each  grade/age  group,  because 
each  of  these  can  be  viewed  as  separate  samples  of  the  appropriate 
population. 
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For  the  spiral  assessment,  13  subgroups  were  defined  in  terms  of  race, 
ethnicity,  Office  of  Business  Economics  region  and  community  size  (SDOC)  as 
shown  in  Table  14.1.     Each  of  the  13  subgroups  was  further  divided  into  three 
classes : 

1)  students  eligible  by  both  age  and  grade; 

2)  students  eligible  by  age  only; 

3)  students  eligible  by  grade  only. 

This  resulted  in  39  poststratif ication  cells  for  each  age  class.  The 
final  weight  for  a  student  is  the  product  of  the  base  weight  (as  adjusted  for 
nonresponse  and  after  trimming)  and  a  poststratif ication  factor  whose 
denominator  is  the  sum  of  those  weights  for  the  cell  to  which  the  student 
belongs  and  whose  numerator  is  an  adjusted  estimate,  based  on  more  reliable 
data,  of  the  total  number  of  students  in  the  cell. 

The  adjusted  estimate  of  the  total  number  of  students  in  a  given  cell  is 
a  composite  of  estimates  from  the  1986  NAEP  sample  and  independent  estimates 
based  on  projections  based  on  1983  and  1984  Current  Population  Survey 
estimates  and  1985  Census  projections.     The  adjusted  estimate  is  a  weighted 
mean  of  the  various  estimates,  the  weights  being  inversely  proportional  to 
the  approximate  relative  variances  of  the  NAEP  and  these  independent 
estimates.     (Further  details  are  provided  in  Burke  et  al. ,  1987). 

The  sample  of  students  in  each  of  the  tape  assessments  was  much  smaller 
than  the  sample  for  the  spiral  assessments.     Consequently,  some  subgroups 
were  collapsed  for  poststratif ication  as  follows: 

1,  2  6,  7 

3  8,  9 

4  10,  11,  12 

5  13 

Furthermore,  there  was  no  subdivision  into  eligibility  classes,  so  that 
there  were  eight  poststratif ication  cells  for  each  age  class.     For  the  Bridge 
assessments,  the  numerators  of  the  poststratif ication  factors  for  these  cells 
were  the  corresponding  adjusted  estimates  used  for  computing  the  spiral 
poststratif ication  factor.     For  each  of  the  tape  assessments  in  Bridge  A  and 
Bridge  B,  the  denominators  were  the  sums  of  the  weights  for  each  age  class. 


14.1.5    The  Final  Student -Weight:     The  Full -Sample  Weight 

The  final  weight  assigned  to  a  student  is  the  student  full-sample 
weight.     This  weight  is  the  student's  base  weight  after  the  application  of 
the  various  adjustments  described  above.     The  student  full-sample  weight  was 
used  to  derive  all  estimates  of  population  and  subpopulation  characteristics 
that  have  been  presented  in  the  various  NAEP  reports,  including  simple 
estimates  suc^;  as  the  proportion  of  students  of  a  specified  type  who  would 
respond  in  a  certain  way  to  an  exercise  and  more  complex  estimates  such  as 
mean  proficiency  levels. 
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Table  14.1 
Major  Subgroups  for  Poststratif ication 


Subgroup 

Race 

Ethnicity 

Region 

SDOC* 

1 

White 

Non-Hispanic 

NE 

1.  2 

2 

White 

Non-Hispanic 

NE 

3,  4,  5 

3 

White 

Non-Hispanic 

SE,  Central 

1,  2 

4 

White 

Non-Hispanic 

SE,  Central 

3 

5 

White 

Non-Hispanic 

SE,  Central 

4,  5 

6 

White 

Non-Hispanic 

West 

1,  2 

7 

White 

Non-Hispanic 

West 

3,  4.  5 

8 

Any 

Hispanic 

NE,SE, Central 

Any 

9 

Any 

Hispanic 

West 

Any 

10 

Black 

Non-Hispanic 

NE 

Any 

11 

Black 

Non-Hispanic 

SE 

Any 

12 

Black 

Non-Hispanic 

Central,  West 

Any 

13 

Other 

Non-Hispanic 

Any 

Any 

*SDOC  (Sample  Description  of  Community)  categories:  l--BLg  City;  2- -Fringe  of 
Big  City;  3— Medium  City;  4--Small  Place;  and  5--Extreme  Rural.  The 
definition  of  the  categories  is  based  on  county  level  1980  Population  Census 
data. 
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14.1.6    Other  Weights 


In  addition  to  the  weights  for  the  assessed  students,  weights  were  also 
derived  for  excluded  students  and  for  the  students  whose  teachers 
participated  in  the  Teacher  Survey. 

Excluded  students  are  those  students  who  were  unable  to  complete  the 
assessment  because  of  being  non- English  speaking,  educable  mentally  retarded, 
or  functionally  disabled.  Weights  for  excluded  students  were  computed 
separately  for  the  students  excluded  from  any  of  the  sessions  in  the  spring 
assessment  (the  spiral  and  Bridge  B  samples  combined)  and  for  the  students 
excluded  from  the  Bridge  A  assessments  (separately  for  the  age  9  and  the  age 
13  samples).  As  in  the  case  of  the  weights  for  the  assessed  students,  the 
excluded  student  weights  were  based  on  the  probability  of  selection  with 
adjustments  for  nonresponse  and  trimming  of  excessively  large  weights. 
Further  details  on  the  derivation  of  the  excluded  student  weights  can  be 
found  in  Burke  et  al.  (1987). 

In  every  spiral -allocated  school,  a  subsample  of  non-excludad  students 
selected  for  spiral  assessment  was  taken.     For  each  subsampled  student 
enrolled  in  a  course  in  the  learning  area  randomly  assigned  to  the  school, 
the  teacher  of  that  course  was  identified  and  asked  to  complete  the  teacher 
questionnaire.     The  responses  of  the  teacher  to  thau  questionnaire  are  tied 
to  the  student  to  enable  the  estimation  of  the  number  or  percent  of  students 
in  the  population  whose  teachers  have  certain  characteristics.  These 
estimates  used  the  teacher/student  weights,  which  are  based  on  the  final 
student  weights  of  those  spiral -assessed  students  who  are  linked  to  a 
completed  teacher  questionnaire.  The  teacher/student  weight  is  the 
nonresponse-adjusted  student  weight  further  adjusted  for  the  probability  that 
the  student's  teacher  was  selected  as  well  as  for  nonresponse  on  the  part  of 
the  teachers.     Additionally,  the  teacher/student  weights  were  subjected  to 
the  trimming  algorithm  and  poststratif ication  adjustments.     Further  details 
on  the  construction  of  these  weights  appears  in  Burke  et  al.  (1987). 

Finally,   in  addition  to  these  weights,  which  were  used  to  derive  all 
estimates  of  population  and  subpopulation  characteristics,  other  sets  of 
weights,  called  jackknife  replicate  weights,  were  derived  to  facilitate  the 
estimation  of  sampling  varialility  by  the  jackknife  variance  estimation 
technique.     These  weights  and  the  jackknife  estimator  are  discussed  in  the 
next  section. 


14.2      PROCEDURES  USED  BY  NAEP  TO  ESTIMATE  SAMPLING  VARIABILITY 

A  major  source  of  uncertainty  in  the  estimation  of  the  value  in  the 
population  of  a  variable  of  interest  exists  because  information  about  the 
variable  is  obtained  on  only  a  sample  from  the  population.     To  reflect  this 
fact,   it  is  important  to  attach  to  any  statistic  (e.g.,  a  mean)  an  estimate 
of  the  sampling  variability  to  be  expected  for  that  statistic.     Estimates  of 
sampling  variability  provide  information  about  how  much  the  value  of  a  given 


-282- 

2Sj 


statistic  would  be  likely  to  change  if  the  statistic  had  been  based  on 
another,  equivalent,  sample  of  individuals  drawn  in  exactly  the  same  manner 
as  the  achieved  sample.     Consequently,  the  estimation  of  the  sampling 
variability  of  any  statistic  must  take  into  account  the  sample  design. 

As  we  have  noted  before,  the  NAEP  sample  is  obtained  via  a  stratified 
multistage  probability  sampling  design  that  includes  provisions  for  sampling 
certain  subpopulations  at  higher  rates.    Additional  chara»^teristics  of  the 
sample  include  adjustments  for  both  nonresponse  and  poststratif ication.  The 
resulting  sample  has  different  statistical  characteristics  from  those  of  a 
simple  random  sample.     In  particular,  because  of  the  effects  of  cluster 
selection  (students  within  schools,  schools  within  PSUs)  and  because  of 
effects  of  nonresponse  and  poststratif ication  adjustments,  observations  made 
on  different  students  cannot  be  assumed  to  be  independent  of  each  other  (and 
are,  in  fact,  generally  positively  correlated).     Furthermore,  to  account  for 
the  differential  probabilities  of  selection  (and  the  various  adjustments), 
each  student  has  an  associated  sampling  weight,  which  must  be  used  in  the 
computation  of  any  statistic  and  which  is  itself  subject  to  sampling 
variability.     Treatment  of  the  data  as  a  simple  random  sample,  with  disregard 
for  the  special  characteristics  of  the  NAEP  sample  design,  will  produce 
underestimates  of  the  true  sampling  variability. 

14.2.1    Estimation  of  the  Sampling  Variance  of  any  Statistic  by 
the  Jackknife 

This  section  describes  how  the  sampling  variability  of  statistics  based 
on  the  NAEP  data  was  estimated.     (The  estimation  of  variability  due  no 
imperfect  measurement,  a  component  of  the  overall  variance  of  NAEP  scale- 
scores,  is  discussed  in  section  8.4). 

The  jackknife  procedure  has  a  number  of  properties  that  make  it 
particularly  suited  for  the  analysis  of  NAEP  data: 

1)  It  approximately  estimates  the  sampling  error  arising 
from  the  complex  sample  selection  and  estima';ion 
procedures  used  in  NAEP. 

2)  It  reflects  the  component  of  sampling  error  introduced  by 
the  use  of  weighting  factors  that  are  dependent  upon  the 
sample  data  actually  obtained. 

3)  It  can  be  adapted  readily  to  the  estimation  of  sampling 
errors  for  parameters  estimated  using  statistical 
modeling  procedures,  as  well  as  for  tabulation  estimates 
such  as  totals  and  means. 

4)  Once  appropriate  weights  are  derived  and  attached  to  each 
record,  jackknifing  is  straightforward  to  use  for 
estimating  sampling  errors.    A  single  set  of  replicate 
weights  is  required  for  all  tabulations  and  model 
parameter  estimates  that  may  be  needed. 
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The  method  of  applying  the  jackknife  procedure  involves  first  defining 
groups  of  pairs  of  first-stage  sampling  units.     For  the  1986  NAEP  spiral 
assessment,  Westat  defined  38  groups  of  such  first-stage  unit  pairs,  where 
pairing  was  performed  so  that  the  populations  represented  by  each  member  of 
the  pair  were  similar.     In  the  definition  and  pairing  of  groups  of  first- 
stage  units,  a  distinction  was  made  between  the  eight  largest  certainty  PSUs, 
the  26  smaller  certainty  PSUs,  and  the  60  noncertainty  PSUs.  The 
noncertainty  PSUs  were  formed  into  30  pairs.    The  pairs  were  formed  on  the 
basis  of  a  number  of  characteristics  of  the  strata  from  which  they  were 
selected,  these  being  the  change  in  population  between  the  1970  and  1980 
population  censuses,  the  proportions  of  the  1980  population  who  were  Black  or 
Hispanic  and  proportions  of  urban  and  farm  populations.     PSUs  from  similar 
strata  were  paired  so  as  to  minimize  the  bias  of  variance  estimation  (which 
reflects  the  between- stratum  variance  within  these  pairs,  even  though  this 
does  not  contribute  to  the  true  sampling  error) .     The  30  pairs  were  then 
combined  into  15  groups  of  pairs,  two  pairs  per  group.     This  combining  of 
pairs  reduced  the  task  of  jackknife  variance  estimation  to  a  more  manageable 
level,  while  not  adding  bias  to  the  variance  estimation.     The  26  smaller 
certainty  PSUs  were  paired  on  the  basis  of  the  same  criteria  used  to  pair  the 
strata  of  noncertainty  PSUs,  plus  the  level  of  educational  expenditure  per 
student.     Each  of  the  eight  largest  certainty  PSUs  constituted  at  least  one 
jackknife  pair  in  itself,  each  member  of  the  pair  consisting  of  schools 
within  the  PSU. 

The  end  result  was  38  jackknife  pairs  of  first-stage  units.  Similar 
pairings  were  defined  for  the  bridge  assessments.     For  the  Bridge  A 
assessment,  there  are  33  pairs;  there  are  also  33  pairs  for  the  Bridge  B 
assessment,  but  these  correspond  to  somewhat  different  clusters  of  first- 
stage  units.     Further  information  on  the  construction  of  jackknife  pairs  can 
be  found  in  Burke  et  al.(1987). 

The  component  of  the  sampling  variability  attributable  to  the  sampling 
of  the  portion  of  the  population  represented  by  a  jackknife  pair  is  estimated 
as  the  squared  difference  between  the  value  of  the  statistic  for  the  complete 
sample  and  a  replicate  estimate  formed  by  recomputing  the  statistic  on  a 
specially  constructed  pseudoreplicate .     This  pseudoreplicate  is  created  from 
the  original  dataset  by  eliminating  one  member  of  the  jackknife  pair  and 
replacing  it  with  a  copy  of  the  data  from  the  complementary  member  of  the 
jackknife  pair.     For  computational  purposes,  the  pseudoreplicate  associated 
with  a  given  pair  is  the  original  dataset  with  a  different  set  of  weights, 
referred  to  as  the  student  replicate  weights.    We  shall  denote  these 
replicate  weights  as  SRWTOl  thr^jigh  SRWT38,  where  SRWTi  is  formed  by  making 
the  above  adjustment  with  the  i      jackknife  pair.     This  set  of  weight  allows 
measurement  of  the  total  effect  of  replacing  one  member  of  the  jackknife  pair 
with  a  copy  of  the  other,  including  adjustments  for  nonresponse  a^ 
poststratif ication.     The  replicate  estimate  associated  with  the  i 
pseudoreplicate  for  a  given  statistic  is  obtained  by  recalculating  the 
statistic  using  the  weights  SRWTi  instead  of  the  student  full-sample  weights. 
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The  student  replicate  weight,  SRWTi,  for  the  i      pseudoreplicate  was 
computed  as  follows : 

Let       be  the  base  weight  of  a  student,  where  the  base  weight  accounts 
for  the  probabilities  of  selection  but  does  not  include  nonresponse  or 
poststratif ication  adjustments. 

Then    SRWTi    -    f.     W„ . 


where 


if  the  student  is  in  the  first  set  of 

first- stage  units  in  jackknife  pair  i 

if  the  student  is  in  the  second  set  of 

first-stage  units  in  jackknife  pair  i 

if  the  student  is  in  neither  of  the 

first- stage  units  in  jackknife  pair  i 

is  the  replicate  base  weight  formed  by  replacing  the  second  member  of  the 
pair  by  the  first,  JF  is  a  constant  multiplier  (usually  equal  to  2)  designed 

PS 

to  maintain  certain  population  totals,  f^      is  the  poststratif ication 

adjustment  factor  based  on  these  replicated  base  weights,  and  f^^  denotes  the 
nonresponse  adjustment  factor  appropriate  for  the  student.     These  replicate 
weights  allow  the  estimation  both  of  the  effect  of  poststratif ication,  and, 
except  for  the  case  of  the  8  largest  certainty  PSUs,  the  effect  of  the 
nonresponse  adjustments.     The  nonresponse  adjustments  are  taken  into  account 
implicitly  because  they  were  performed  within  the  76  jackknife  pair  halves, 
except  in  the  case  of  the  8  certainty  PSUs,  where  adjustments  were  made 
within  PSU,  but  across  the  two  pair  halves. 

As  a  specific  example  of  the  use  of  the  student  replicate  weights,  let 
t(2,!£)  be  any  statistic  that  is  a  function  of  the  sample  responses  y  and  the 
weights  w  and  that  estimates  population  value  T.     For  example,   t  could  be  a 
weighted  mean,  a  weighted  percent-correct  point  or  a  weighted  regression 
coefficient.     The  t(x,w),  computed  with  the  sampling  weights  (WEIGHT  on  the 
data  tapes)  is  the  appropriate  sample  estimate  of  T.     To  compute  Var(t),  the 
sampling  variance  for  this  statistic,  proceed  in  the  following  manner: 

1)  For  each  of  the  38  pairs  of  first-stage  units,  compute 
the  aggociated  pseudoreplicate  for  the  statistic.  For 
the  i      pair,  this  is 

t^  -  t(x, SRWTi) 

which  is  the  statistic  t  recalculated  by  using  SRWTi  instead 
of  the  sampling  weights. 


W  ' 
^Bi  ^ 


NR 

JF  *  f^"^  *  W. 


f^^*  W. 
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The  sample  variance  of  t  is 


Var(t)  «  E 
i«l 


2 

(t.  .  t)^ 


This  estimation  technique  is  called  the  multiweight  jackknife  approach 
and  was  used  by  NAEP  to  estimate  all  sampling  errors  presented  in  the  various 
reports.     A  similar  procedure  was  followed  to  estimate  the  sampling 
variability  for  statistics  based  on  any  of  the  bridge  samples.     The  only 
difference  was  in  the  number  of  jackknife  pairs  (and  hence  replicate  weights) 
used. 

A  further  discussion  of  the  variance  estimation  procedure  used  by  ETS 
including  a  discussion  of  alternative  jackknife  estimators  that  were  also 
considered  appears  in  Johnson  (1987a). 

14.2.2  The  Degrees  of  Freedom  of  the  Variance  Estimate 

Note  that  the  jackknife  procedure  estimates  the  sampling  variability  of 
the  statistic  by  assessing  the  effect  of  change  in  the  sample  at  the  level  of 
clusters  of  first-stage  units.     For  this  reason,  the  number  of  degrees  of 
freedom  of  the  variance  estimate  Var(t)  will  be  at  most  equal  to  the  number 
of  pairs.     The  number  of  degrees  of  freedom,  which  is  indicative  of  the 
variability  of  the  variance  estimate,  equals  the  number  of  independent  pieces 
of  information  used  to  generate  the  variance  estimates.     In  the  current  case, 
for  the  spiral  sampl^,  the  pieces  of  information  are  the  38  squared 
differences  (t.   -  t)   , which  are  approximately  independent,  each  supplying  at 
most  one  degree  of  freedom,  regardless  of  how  many  individuals  were  sampled 
within  any  PSU.     (There  are  fewer  squared  differences  with  the  bridge 
samples.)     In  fact,  as  shown  in  Johnson  (1988),  there  can  be  considerably 
fewer  than  38  degrees  of  freedom  attributable  to  jackknife  variance  estimate 
for  the  NAEP  sample. 

14.2.3  Approximating  the  Sampling  Variance  Using  Design  Effects 

The  major  computational  load  in  computing  uncertainty  measures  for  any 
statistic  exists  in  the  computation  of  the  uncertainty  due  to  sampling 
variability.     As  noted  in  section  14.2.1,  the  jackknife  estimate  of  the 
variability  of  a  statistic  based  on  one  or  more  observed  NAEP  variables  in 
the  1986  spiral  sample  requires  computing  the  statistic  39  times.  Because 
the  cost  of  the  full  procedure  may  well  prove  prohibitive  in  many  studies,  an 
approximate  procedure  that  can  produce  reasonable  approximations  at  lower 
costs  is  provided  in  this  section. 

As  indicated  in  section  14.2.1,  it  is  inappropriate  to  estimate  the 
sampling  variability  of  any  statistic  based  on  the  NAEP  database  by  using 
simple  random  sampling  (SRS)  formulas.     These  formulas,  which  are  the  ones 
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used  by  most  standard  statistical  software  such  as  SPSS  and  SAS,  will  produce 
variance  estimates  that  are  generally  much  smaller  than  is  warranted  by  the 
sample  design. 

It  may  be  possible  to  account  approximately  for  the  effects  of  the 
sample  design  by  using  an  inflation  factor,  the  design  effect,  developed  by 
Kish  (1965)  and  extended  by  Kish  and  Frankel  (1974).     The  design  effect  for  a 
statistic  is  the  ratio  of  the  actual  variance  of  the  statistic  (taking  the 
sample  design  into  account)  to  the  conventional  variance  estimate  based  on 
the  same  number  of  elements.     To  avoid  sources  of  bias  due  to  improper 
representation,  this  conventional  estimate  must  use  the  sampling  weights. 
The  design  effect  may  be  used  to  adjust  error  estimates  based  on  simple 
random  sampling  assumptions  to  account  approximately  for  the  effect  of  the 
design.     In  practice,  this  is  often  accomplished  by  dividing  the  total  sample 
size  by  the  design  effect  and  using  this  effective  sample  size  in  the 
computation  of  errors.     Note  that  the  value  of  the  design  effect  depends  on 
the  type  of  statistic  computed  and  the  variables  considered  in  a  particular 
analysis  as  well  as  the  clustering  effects  occurring  among  sampled  elements. 

Further  discussions  on  the  derivation  and  characteristics  of  design 
effects  can  be  found  in  Kish  (1965),  Kish  and  Frankel  (1974),  and  in  Johnson 
(1987b) . 

A.^.  an  example  of  the  distribution  of  design  effects  to  be  expected  from 
the  198S  NAEP  data,  we  consider  the  design  effect  for  the  key  statistic,  P, 
the  estimated  proportion  of  a  specified  subgroup  of  the  population  who  would 
corre.;tly  respond  to  a  given  assessment  exercise.     This  estimate,  which  is  a 
weighted  mean  of  the  responses  of  individuals  in  the  subgroup  to  the  exercise 
(where  an  individual's  response  is  either  0  or  1) ,  has  a  design  effect  of  the 
form 

deff(P)  -  Varj^(P)/(P(l  -  P)/N) 

In  the  above,  N  is  the  total  number  of  individuals  in  the  subgroup  responding 
to  the  exercise,  Varj^(P)  is  the  jackknife  variance  of  P,  and  P(l  -  P)/N  is 
the  conventional  variance  estimate  of  P.     (The  estimate  P(l  -  P)/N  has  the 
same  form  as  the  simple  random  sampling  estimator  of  the  variance  of  P.  In 
fact,  the  sample  weights  have  been  taken  into  account  via  the  weighted 
estimation  of  P. ) 

The  distributions  of  design  effects  for  proportions  correct  by  grade  and 
by  demographic  subgroup  within  grade  across  all  cognitive  reading  items 
presented  in  the  1986  spiral  assessment  are  indicated  in  Tables  14.2  through 
14.4. 

Table  14.2  addresses  the  distributions  of  the  design  effects  for  the  63 
multiple-choice  cognitive  reading  exercises  preseiited  in  1986  to  grade  3 
students.     These  distributions  are  shown  for  the  population  as  a  whole 
("total")  as  well  as  for  a  variety  of  demographic  subgroups:  sex; 
race/ethnicity  (White,  Black,  Hispanic,  other);  age  (less  than  modal  age, 
modal  age,  greater  than  modal  age);  region  (Northeast,  Southeast,  Central, 
West);     Size  and  Type  of  Community  (Rural,  Low  Metropolitan,  High 
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Table  14.2 


Distribution  of  Design  Effects 
by  Demographic  Subgroup 
for  the  Cognitive  Reading  Items  Given  in  1986 

Grade  3* 


Group 

jLoy 

1  IcU- JLdll 

HiO 

Max 

iOiAL 

i.  ,  /  J 

3. 15 

1.  75 

MALE 

1  .  JO 

i.  . 

9  m 

2  94 

1.  73 

'CTT'XI  A  T  t? 

rEMALE 

1  .  i-O 

1  "^7 

L  .  D  / 

1  79 

2  69 

1.49 

TTLJT'Ft? 
WHiiE 

1    9  A 

^  A.ft 

3 .01 

1.59 

T5  T  A  /^V 

BLACK 

1  in 

1  .  lU 

i.  .  DO 

1  70 

3 . 22 

1.47 

1  HA 

i.  .  JO 

1  S9 

2.  32 

1.37 

UinEK 

L  .  /  D 

3  23 

8 . 57 

2.56 

<  AbE 

L  .  ID 

1  71 

2.63 

1 .47 

—  Apt? 

i.  .  J  J 

1  70 

1.94 

3  .08 

1.73 

APT? 

1  9ft 

J. .  J 

1 . 91 

2.95 

1.62 

2.00 

4.79 

1.66 

1. 36 

1.96 

3.36 

1.47 

CENTRAL 

1.05 

1.34 

2.00 

4.70 

1.53 

WEST 

1.19 

2.12 

2.81 

5.75 

2.16 

RURAL 

1.08 

1.63 

1.94 

5.02 

1.69 

LOW  MET 

1.04 

1.40 

1.77 

3.22 

1.45 

HI  MET 

1.10 

1.35 

1.64 

2.91 

1.42 

BIG  CITY 

1.15 

1.52 

2.07 

4.04 

1.65 

FRINGE 

1.02 

1.37 

1.54 

4.09 

1.42 

MED  CITY 

1.12 

1.69 

2.37 

4.18 

1.77 

SMALL  PL 

1.16 

1.51 

2.14 

3.49 

1.70 

<  H.S. 

1.09 

1.36 

1.59 

2.30 

1.36 

GRAD  HS 

1.05 

1.41 

1.73 

2  91 

1.45 

POST  HS 

1.02 

1.23 

1.73 

2.61 

1.39 

GRAD  COL 

1.02 

1.21 

1.46 

2.61 

1.29 

UNKNOWN 

1.41 

1.57 

1.92 

3.50 

1.68 

*  Distributions  based  on  63  multiple -choice  items 


ERIC 


-288-  2?:; 


Table  14.3 


Distribution  of  Design  Effects 
by  Demographic  Subgroup 
for  the  Cognitive  Reading  Items  Given  in  1986 

Grade  7* 


Group 

LoO 

Median 

HiQ 

1 

.22 

1 

.51 

1 . 80 

MAT  17 

1 

.21 

1 

.49 

1 .73 

FFMAT  F 

1 

.06 

1 

.32 

1 .48 

UHTTF 

1 

.12 

1 

.40 

1 . 72 

1 

.17 

1 

.47 

1 . 64 

HT<?PANTr! 
11 X  o  X  rvLi  X  \j 

1 

.19 

1 

.57 

1 .92 

OTHER 

1 

.21 

1 

.75 

Z .  2o 

1 

.07 

1 

.27 

1 . 50 

AriF 

1 

.14 

1 

36 

1.31 

>  AfiF 

JTiXJ  ill 

1 

.19 

1 

34 

1 .56 

NF 

0 

.84 

1 

22 

1 .  /3 

SE 

0 

.92 

1 

21 

1  .  J  o 

CENTRAL 

0 

.80 

1 

52 

2.20 

WEST 

1 

.10 

1 

44 

2.30 

RURAL 

1 

.07 

1 

62 

2.23 

LOW  MET 

1 

.19 

1 

54 

1.99 

HI  MET 

1 

.02 

1 

30 

1.60 

BIG  CITY 

1 

.19 

1 

48 

1.84 

FRINGE 

1 

.17 

1 

40 

1.78 

MED  CITY 

1 

.04 

1 

34 

1.83 

SMALL  PL 

0 

.94 

1 

14 

1.77 

<  H.S. 

1 

.20 

1 

46 

1.86 

GRAD  HS 

1 

.22 

1 

45 

1.84 

POST  HS 

0 

94 

1 

18 

1.48 

GRAD  HS 

1 

i:. 

1 

33 

1.59 

UNKNOWN 

1 

.14 

1 

33 

1.62 

Max 

Mean 

3.42 

1.56 

3.39 

1.58 

2.33 

1.31 

3.54 

1.46 

2.68 

1.43 

2.96 

1.60 

11.15 

2.01 

2.32 

1.31 

3.25 

1.47 

2.94 

1.40 

2.87 

1.33 

2.67 

1.26 

5.26 

1.70 

5.80 

1.74 

3.81 

1.67 

4.54 

1.65 

2.45 

1.34 

2.66 

1.51 

3.93 

1.56 

3.95 

1.47 

3.77 

1.39 

3.12 

1.55 

3.42 

1.53 

2.37 

1.27 

2.62 

1.35 

2.61 

1.42 

*  Distribution  based  on  65  multiple-choice  items 
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Table  14.4 


Distribution  of  Design  Effects 
by  Demographic  Subgroup 
for  the  Cognitive  Reading  Items  Given  in  1986 

Grade  11* 


Group 

LoU 

Median 

riax 

Mean 

TOTAL 

1  Aft 

9  07 

1 . 78 

MALE 

1     O  O 

1 .  Dy 

1  QQ 

so 

1  71 

FEMALE 

1  r\Q 

1.^1 

1  ,  /  u 

OS 

1  44 

WHITE 

1  R7 
1 .  D  / 

1  Q7 
1.7/ 

A  ^^9 

1  Ifi 

BLACK 

1 .  Z  J 

1  A9 
1  . 

9  77 

1  '^7 

JL  .  / 

HISPANIC 

1  17 
1.1/ 

1 .  J  3 

1  Sft 

9  1 

1  '^7 

JL  ■  ^  / 

OTHER 

n  QQ 

1  ^^7 
1  .  J  / 

1  Ql 

J-  .  7  J- 

90 

1 .47 

<  AGE 

1  .  UO 

1  97 

1  S4 
J. ,  j*+ 

9  8  S 

1 . 30 

1 .  lo 

1  69 
1.  .  (J  ^ 

1 . 98 

3 .41 

1.  67 

A  CXt 

1     1  Zl 
L  .  LM- 

1.61 

2.49 

1.39 

1  HZl 
I.  .  U*+ 

1  SS 

9  17 

5.99 

1.  83 

CT7 

u  .  /  o 

1.83 

3.33 

1.40 

CENTRAL 

1.07 

1.71 

2.55 

6.13 

2.05 

WEST 

1.05 

1.29 

2.12 

5.47 

1.67 

RURAL 

0.85 

1.48 

1.93 

4.62 

1.58 

LOW  MET 

1.14 

1.46 

1.90 

4.94 

1.65 

HI  MET 

1.03 

1.26 

1.76 

2. 66 

1.38 

BIG  CITY 

0.97 

1.21 

1.74 

4.45 

1.45 

FRINGE 

0.83 

1.29 

1.69 

3.28 

1.41 

MED  CITY 

1.16 

1.50 

2.18 

4.17 

1.74 

SMALL  PL 

1.13 

1.57 

1.92 

3.85 

1.61 

<  H.S. 

1.17 

1.33 

1.60 

3.09 

1.44 

GRAD  HS 

1.12 

1.35 

1.60 

2.86 

1.41 

POST  HS 

1.02 

1.26 

1.64 

2.66 

1.41 

GRAD  COL 

1.24 

1.58 

1.97 

3.77 

1.69 

UNKNOWN 

1.03 

1.17 

1.39 

2.07 

1.23 

*  Distributions  based  on  65  multi.ple -choice  items 
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Metropolitan.  Big  City,  Urban  Fringe.  Medium  City.  Small  Place);  and  parental 
Graduated  College.  Unknown).    For  each  of  these  groupings  of  grade  3 

ouarMl':*.^'m'  '^"""^  ^""'^^^^  (LoQ).\edial  upper 

quartile  (HiQ)  and  maximum  design  effect  as  well  as  the  mean  design  effect. 

multi^Jfrhi?;!!  information  on  the  distributions  of  design  effects  for  the  65 
multiple-choice  cognitive  reading  exercises  presented  to  grade  7  students 

ore:^t.d^  "^l:"        --^^^^^--^olce  cognitive  heading  items 

presented  to  grade  11  students  are  addressed  by  Table  14.4. 

r.^fo^^  particular  demographic  variables  shown  (sex.  race/ethnicity,  age. 
region,  parental  education,  and  size  and  type  of  community)  were  selected 
because  (1)  they  are  major  variables  in  NAEP  reports  and  (2)  they  reflect 
different  types  of  divisions  of  the  population  that  might  have  different 
levels  of  sampling  variability.  "xi-terenc 

lar^.^^^..'f  1^^-^!?^  figures  show  that  the  design  effects  are  predominantly 
ItJt  11  '   "'^Jt''^'''"^  ^^^"^  standard  variance  estimation  formulas  will  be 

S^ta      L  '  markedly  so.     Although  the  distributions  of 

design  effects  appear  somewhat  different  for  certain  subgroups  of  the 
population,   they  are.  perhaps,  similar  enough  (at  least  within  a  grade)  to 
select  an  overall  composite  value  that  is  adequate  for  most  purposes  In 
choosing  a  composite  design  effect,  some  consideration  must  be  made  about  the 
relative  consequences  of  overestimating  the  variance  as  opposed  to 
o^eresM^tM  T  ^^-Pl-'  adopting  the  position  that  an 

ZT.Zl  variance  is  as  severe  an  error  as  an  underestimate  leads 

hLV    ^rP  "^r^^^f  ^  ?^at  is  near  to  the  center  of  the  distributions  of  the 
des.-fn  IffTrr"-     r^'';"""  --Po-tes  of  this  type  are  the  mean  and  median 
design  effects.     In  the  current  data,   the  mean  design  effects  are  1  8  16 
and  18  for  the  total  populations  in  graues  3.  7  and  11.  respectively'  These 
are  close  to.  but  greater  than,  the  median  design  effects:     1.7.  1.5  and  1.7. 

Alternatively,  one  can  adopt  the  position  that  it  is  a  graver  error  to 
underestimate  the  variability  of  a  statistic  than  to  overestimate  it  For 

T,i7r^l\  ""'"^  ^^^^^^  ^^""^"^  estimation  of  variances  using  design 

effecs    among  other  techniques)  under  assumption  that  the  consequences  of  an 
underestimate  are  three  times  as  severe  as  those  of  an  overestimate  of  the 
same  magnitude.     Assuming  that  the  distribution  of  design  effects  is  roughly 
independent  of  the  jackknife  variance,  so  that  the  size  of  a  design  effect 
does  not  depend  on  the  size  of  the  variance,  and  adopting  a  loss  function 
^cru.1     ^r'^^'"^         °^  ""^""^  °f  '^^^  deviations  of  predicted  from 

produLr^^  ""^""^^^^^"•^^^  receiving  three  times  the  weight  of  overestimates, 
produces  the  upper  quartile  of  the  design  effects  as  the  composite  value 
The  values  of  this  composite,  for  the  total  populations  in  grades  3,   7.  and 
11.  respectively,  are  2.1,  1.8.  and  2.1.  .  . 
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and  Estimates  of  the  Proficiency 
of  American  Students 


Chapter  15 


STATISTICAL  SUMMARY  OF  THE  1986  NAEP  SAMPLE  AND 
ESTIMATES  OF  THE  PROFICIENCIES  OF  AMERICAN  STUDENTS 


Albert  E.  Beaton,  David  S.  Freund, 
Bruce  A.  Kaplan,  and  Michael  A.  Narcowich 

Educational  Testing  Service 


This  chapter  presents  a  statistical  summary  of  the  NAEP  1986  sample  and 
some  selected  results  from  the  assessment. 

This  technical  report  so  far  has  dealt  primarily  with  the  design  of  the 
1986  National  Assessment  and  the  processes  and  procedures  that  were  used  in 
collecting  and  analyzing  the  data.     Chapter  2  provides  an  overview  of  the 
1986  NAEP  design  and  summarizes  how  the  students  were  selected,  how  they  were 
assessed,  and  how  their  responses  moved  from  assessment  sessions  to  a 
carefully  constructed  database,  ready  for  analysis.     Chapter  7  summarizes  the 
methods  of  data  analysis,  including  scaling  and  parameter  estimation.  This 
chapter  will  assume  a  general  familiarity  with  the  structure  of  NAEP  as 
summarized  in  those  chapters . 

In  this  chapter,  three  of  the  many  types  of  NAEP  results  are  presented: 

results  of  the  instrument  development  process,  including 
the  sizes  of  the  items  pools  and  numbers  of  booklets; 

results  of  the  sampling  process,  including  the  numbers  of 
students  in  each  sample  by  selected  subgroups;  and 

results  of  the  parameter  estimation  process,  including 
estimates  of  the  proficiencies  of  several  populations  of 
students  in  reading,  mathematics,  and  science. 

This  is  a  technical  report  and  is  not  Intended  to  be  interpretive. 
Estimates  are  presented,  but  no  attempt  is  made  to  explain  or  evaluate  the 
students'  performance.     Interpretiva  results  are  presented  in  NAEP  reports 
such  as  Who  Reads  Best?  (Applebee,  Langer,  6t  Mullis,  1988)  and  The 
Mathematics  Report  Card:     Are  We  Measuring  Up?  (Dossey,  Mullis,  Lindquist,  6t 
Chambers,  1988).     We  will  leave  it  to  experts  in  the  educational  process  to 
hypothesize  why  the  results  occurred.     The  public -use  data  tapes  and  user 
guide  (Rogers,  Kline,  Norris ,  Johnson,  Mislevy,  Zwick,  Barone,  6c  Kaplan, 
1988)  are  available  for  those  who  wish  to  estimate  other  parameters  of 
student  performance  from  the  NAEP  data  or  to  search  for  possible  explanations 
for  the  population  characteristics  that  are  reported  here. 

Clearly,  neither  this  report,  nor  any  report,  could  present  all  of  the 
population  estimates  that  are  made  possible  by  the  NAEP  database.  The 
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analysis  of  the  1986  NAEP  data  has  resulted  in  the  production  of  many 
thousands  of  tables  containing  estimates  of  the  proficiency  of  students, 
and  various  subgroups  of  students,  in  American  schools.    We  have  selected  a 
few  basic  tables  for  presentation  here.  The  technical  details  of  the 
estimation  process  that  underlies  these  tables  are  covered  in  the  previous 
parts  of  this  report  and  not  repeated  here.     A  detailed  discussion  of  how  to 
read  and  use  these  tables  is  given  by  Zwick  (1987b). 

15.1    Measurement  Instruments 

A  total  of  56  assessment  booklets  and  questionnaires  was  printed  for  age 
class  9,  72  for  age  class  13,  and  97  for  age  class  17.     These  booklets  are 
enumerated  by  age  level  and  by  type  of  measurement  instrument  in  Table  15.1. 
Some  of  the  instrioments  were  used  at  more  than  one  age/grade  level. 

The  item  pool  used  to  develop  these  booklets  is  described  in  Table  15.2. 
In  general,   t-  ^re  are  two  types  of  items,  cognitive  and  noncognitive .  The 
cognitive  items  are  developed  to  measure  proficiency  in  particular  subject 
areas,  such  as  reading  and  mathematics.     Cognitive  items  may  be  open-ended  or 
multiple - choice .     Open-ended  responses  in  reading  and  science  were 
categorized  according  to  guidelines  and  then  the  acceptable  categories  were 
combined  for  the  purpose  of  scaling.    The  multiple-choice  items  were  scored 
right  or  wrong.     The  noncognitive  items  are  usually  questions  about  the 
student's  or  teacher's  backgrounds  and  attitudes  but  may  also  probe  other 
areas  such  as  school  policies  or  teaching  methods.     Noncognitive  items  are 
not  scored  right  or  wrong.     Many  items  were  used  at  several  age  levels,  and 
thus  the  total  number  of  items  in  an  item  pool  is  not  the  sum  of  the  item 
pools  used  for  the  three  age  classes. 

All  of  the  items  in  the  subject  area  pools  were  used  for  the  main  NAEP 
assessment,  but  not  all  could  be  used  for  the  Bridge  A  *"and  Bridge  R 
assessments.     Table  15.3  shows  the  number  cognitive  items  in  each  subject 
area  that  were  used  in  the  separate  samples . 

The  pools  of  items  were  divided  into  subject  area  blocks.     These  blocks 
contain  some  cognitive  and  some  noncognitive  items  from  a  single  subject 
area.     All  subject  area  blocks  within  an  age  class  were  designed  to  take  the 
same  amount  of  time  for  a  student  to  complete.     The  blocks  for  age  class  9 
were  expected  to  take  13  minutes  and  the  blocks  for  age  classes  13  and  17 
were  expected  to  take  16  minutes  for  a  student  to  complete. 

The  subject  area  blocks  were  then  assigned  to  student  assessment 
booklets.     Each  booklet  for  the  main  NAEP,  Bridge  A,  Bridge  B,  and  Language 
Minority  Probe  student  samples  contained  four  sections,  including  a  common 
block  and  three  variable  subject  area  blocks.     The  common  block  contained 
only  noncognitive  items.     The  three  variable  blocks  in  a  booklet  might  be  all 
from  a  single  subject  area  (e.g.,  reading)  or  from  two  or  three  different 
subject  areas. 

The  other  questionnaires  (excluded  student,  teacher,  school,  and 
computer  coordinator)  contained  only  noncognitive  questions.     For  the 
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questionnaires,  the  number  of  items  in  the  noncognitive  pools  are  the  same  as 
the  number  of  items  in  the  questionnaires. 

More  information  about  the  instruments  that  were  developed  is  contained 
in  Chapters  2  and  4  and  in  Appendix  A.     Tables  A.l  though  A. 3  show  how  the 
variable  blocks  were  assigned  to  booklets  for  the  various  age  classes. 
Tables  A. 4  through  A. 6  depict  the  blocks  with  which  each  variable  block  was 
paired.     Note  that  each  block  was  paired  with  each  other  block  within  its 
subject  area,  but  only  with  some  blocks  in  the  other  subject  areas.  Tables 
A. 7  through  A. 9  show  hov  many  cognitive  and  noncognitive  items  were  in  each 
block,  as  well  as  the  number  of  open-ended  items.     Tables  A. 10  through  A. 12 
show  the  booklets  in  which  each  subject  area  block  was  placed.     Tables  A. 13 
through  A. 25  list,  by  subject  area,  each  cognitive  and  noncognitive  item  and 
the  block  in  which  it  occurred. 


15.2    Sample  Characteristics 

In  this  section,  the  characteristics  of  the  final  NAEP  sample  will  be 
described.     The  process  by  which  the  sample  was  selected  is  discussed  in 
Chapter  3. 

In  the  1986  assessment,  NAEP  contacted  2,095  schools,  of  which  1,633 
contributed  data  to  the  assessment.     The  disposition  of  these  schools  is 
shown  in  Table  15.4.  Some  of  the  schools  were  unwilling  to  cooperate  and 
others  were  believed  to  be  eligible  from  the  sampling  frame,  but  were  not. 
The  cooperation  rate  is  calculated  as  the  sum  of  cooperating  school  and  the 
schools  which  were  found  to  have  no  eligible  students  divided  by  the  same  sum 
plus  the  schools  that  refused  or  were  from  districts  that  refused  to 
cooperate . 

Table  15.4  also  shows  the  number  of  schools  in  several  categories: 
region  of  the  country  (northeast,  southeast,  central,  west),  school 
governance  (public,  private.  Catholic,  Bureau  of  Indian  Affairs,  Department 
of  Defense),  size  and  type  of  community,  degree  of  urbanicity,  grade  span  of 
school,  number  of  teachers,  and  number  of  students. 

The  number  of  respondents  to  the  teacher  questionnaire  is  summarized  in 
Table  15.5.     The  first  column  in  this  table  includes  the  number  of  teachers 
who  responded  by  age  class  and  subject  area.     The  second  column  contains  the 
number  of  teachers  for  whom  at  least  one  student  is  available  in  the  sample 
(see  Chapter  4  for  details  of  the  administration  of  the  teacher 
questionnaire).     The  final  column  contains  the  number  of  students  whose 
teachers  responded  to  the  questionnaire. 

NAEP  is  administered  in  units  called  assessment  sessions.     If  the  number 
of  students  attending  an  assessment  session  is  less  than  a  predetermined 
number  (see  Caldwell  &  Slobasky,  1988),  a  makeup  session  is  held  to  which  the 
missing  students  are  assigned  and  then  assessed.     Table  15.6  shows  the  number 
of  regular  and  makeup  sessions  in  1986  NAEP  by  age  class  for  the  main  NAEP 
and  two  bridge  samples. 
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Altogether,  119,137  students  were  involved  in  1986  NAEP,  including 
excluded  students.     The  breakdown  by  age  class  and  by  sample  is  shown  in 
Table  15.7. 

Tables  15.8  through  15.10  display  the  distribution  of  the  students  in 
several  basic  categories  for  the  three  age  classes:     sex,  racial/ethnic 
grouping,  region  of  the  country,  parental  education,  and  size  and  type  of 
conimunity.     These  tables  have  four  columns: 


eligible  by  age,  which  means  that  the  students  were  in  an 
apv'T-opriate  age  group; 

eligible  by  grade,  which  means  that  the  students  were  in 
an  appropriate  grade; 

eligible  by  age  and  by  grade,  which  means  that  the 
students  were  of  both  an  appropriate  age  and  appropriate 
grade ;  and 

eligible  by  age  or  by  grade,  which  is  the  total  number  of 
students  whose  data  were  collected. 


Table  15.11  and  15.12  contain  the  distribution  of  students  in  the  same 
categories  by  age  class  for  the  two  bridge  samples.  Bridge  sample  students 
were  sampled  by  age  only. 

Tables  15.13  through  15.15  contain  the  distribution  of  excluded  students 
by  age  class.     The  first  columns  categorizes  the  excluded  students  in  the 
Bridge  A  samples,  which  were  age-only  samples.     The  remaining  four  columns 
categorize  the  excluded  students  in  both  the  main  NAEP  and  the  Bridge  B 
samples.     These  student  numbers  are  reported  separately  by  age  eligibility, 
grade  eligibility,  age-and-grade  eligibility,  and  age-or-grade  eligibility. 


15.3    Population  Estimates 

The  1986  NAEP  samples  were  designed  for  estimating  the  size  and  certain 
attributes  of  a  number  of  populations  of  students.     The  estimation  procedures 
use  sampling  weights,  developed  by  Westat,  Inc.,  that  are  used  in  conjunction 
with  the  members  of  the  sample  (see  Chapter  3) .     In  this  chapter,  all 
estimates  of  population  parameters  use  these  sampling  weights. 

Table  15.16  shows  the  sizes  of  the  various  samples  and  the  sums  of  their 
sampling  weights  by  grade/age.     The  sums  of  the  weights  for  the  main  NAEP 
samples,  which  are  by  far  the  largest,  estimate  the  numbers  of  students  who 
are  in  each  grade/age  and  who  would  be  assessable.     The  sum  of  the  weights  of 
the  Bridge  A  and  Bridge  B  samples  estimate  the  number  of  assessable  students 
in  the  various  age-eligible  populations.     The  sums  of  the  weights  of  the 
excluded  students  estimate  the  numbers  of  students  in  each  age  or  grade/age 
combination  who,  in  their  schools'  judgment,  would  not  be  assessabl^^.  The 
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excluded  students  from  the  main  NAEP  and  Bridge  B  samples  are 
indistinguishable,   thus  this  group  of  excluded  students  can  be  combined  with 
the  either  the  main  NAEP  or  Bridge  B  samples  for  total  population  estimation. 
Because  of  changes  in  age  definitions  and  times  of  assessment,   the  Bridge  A 
samples  represent  different  populations. 

In  most  cases,   the  number  of  students  in  a  grade/age  combination  is  not 
of  interest;  a  researcher  will  be  interested  in  estimating  the  number  of 
students  at  either  a  grade  or  an  age  level.       For  the  samples  that  contain 
both  grade-  and  age-eligible  students,  an  estimate  of  the  number  of  students 
at  an  age  level  can  be  made  by  summing  the  weights  of  only  the  age -eligible 
students  and  adding  the  corresponding  sample  of  age-eligible  excluded 
students.     An  estimate  of  the  number  of  students  in  a  grade  sample  can  be 
made  by  summing  the  weights  of  grade-eligible  students  plus  the  weights  of 
grade- eligible  students  from  the  appropriate  excluded  student  sample. 

From  the  main  NAEP  samples,   the  next  tables  estimate  how  many  students 
are  age-eligible  and  grade -eligible  by  age  class.     Tables  15.17  through  15.19 
show  how  many  students  at  a  particular  grade  level  are  at,  in,  or  above  the 
modal  age  for  that  grade,  and  how  many  at  a  particular  age  level  are  at,  in, 
or  above  the  modal  grade  for  that  age.     Along  with  the  counts  from  these 
samples,   the  sum  of  the  weights  (Weighted  N)  for  each  category  is  presented, 
and  these  sums  are  estimates  of  the  numbers  of  students  in  these  categories 
in  the  population.     The  standard  errors  of  these  estimates  and  coefficients 
of  variation  are  also  given. 

Tables  15.20  through  15.22  contain  the  same  type  of  information  for  the 
several  bridge  booklets,  by  age  level.     Since  the  bridge  samples  do  not 
contain  students  who  were  not  age-eligible,   the  sample  is  separated  by  below, 
at,  or  above  modal  grade.     Note  that  booklets  1,  2,  and  3  are  from  Bridge  A 
samples  and  do  not  represent  precisely  the  same  populations  as  booklets  4  and 
5,  which  were  used  in  Bridge  B  samples. 

The  next  tables  show  the  sizes  of  the  estimated  populations  of 
assessable  students  for  various  NAEP  reporting  categories.     These  categories 
include  sex,  racial/ethnic  grouping,  region  of  the  country,  parents' 
education,  and  size  and  type  of  community.     The  estimated  subpopulation  sizes 
for  tlij   main  NAEP  samples  are  shown  in  Tables  15.23  through  15.25,  separately 
by  age  eligibility,  grade  eligibility,  and  grade/age  eligibility.  Tables 
15.26,  15.27,  and  15.28  show  the  same  information  for  the  bridge  samples,  by 
booklet  assignment. 

Tables  15.29  to  15.31  show  the  estimation  total  population  of  excluded 
students  by  demographic  subgroup.     The  first  column  is  from  Bridge  A,  which 
sampled  age-eligible  students  only.     The  next  three  columns  are  derived  from 
main  NAEP  and  Bridge  B  samples,  and  so  separate  age  eligibles  and  grade 
eligibles . 

The  students  in  the  main  NAEP  samples  received  only  three  assessment 
blocks,   thus  they  could  be  assessed  in  one,  two,  or  at  most  three  of  the 
assessment  areas,  but  not  all.     Students  were  assigned  proficiency  values  in 
a  subject  area  only  if  they  received  at  least  one  assessment  block  in  that 


area,  and  thus  the  sample  sizes  of  students  who  have  proficiency  values  vary 
from  one  subiect  area  to  another.     Tables  15.32  through  Table  15.40  show  the 
number  of  students  with  proficiency  values  in  each  subject  are  by  age  and 
grade  combinations.  Tables  15.41  and  15.42  show  history  and  literature  sample 
sizes  for  grade  11/age  17  (the  only  age  class  assessed  in  these  subject 
areas) . 

Tables  15.43  to  15.51  contain  population  estimates  of  student 
proficiencies  by  grade  and  by  subpopulations .     The  subpopulations  are:  the 
sexes,  racial/ethnic  groupings,  ages,  regions  of  the  country,  size  and  type 
of  community,  parental  education,  and  type  of  school.     For  grade  11  students, 
the  results  are  also  presented  by  the  type  of  high  school  prograin.  The 
information  about  proficiency  includes  the  mean  and  standard  deviation  of 
each  subpopulation  as  well  as  the  value  of  the  10th,  25th,  50th  (median), 
75th,  and  90th  percentiles.     Results  are  shown  separately  for  reading, 
mathematics,  and  science.     Only  overall  proficiency  is  shown  for  mathematics  . 
and  science,  although  subscale  estimates  were  computed  for  individuals.  The 
standard  errors  of  the  estimates  are  included  in  parentheses. 

Tables  15.52  through  15.114  contain  results  for  more  finely  defined 
subpopulations.     The  major  reporting  variables  (sex,  race/ethnicity,  parental 
education)  are  cross-classified  with  each  other  and  with  other  reporting 
variables  (region,  articles  in  the  home,  and  television  watching)  to  define 
the  subpopulations.     For  example.  Table  15.58  cross -classifies  sex, 
racial/ethnic  grouping,  and  parental  education  with  the  hours  that  3rd 
graders  spend  watching  television  each  day.     Information  included  about  these 
subpopulations  is  the  actual  sample  size,  the  estimated  population  size  (and 
its  relative  variance) ,  the  proportion  of  students  in  each  subpopulation  (and 
its  standard  error),  and  the  average  proficiency  of  the  students  (and  its 
standard  arror) . 
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Table  15.1 

Measurement  Instruments  Developed  for  1986  NAEP 

 Age  Class- 


9 

13 

17 

MAIN  NAEP  BOOKLETS 

46 

62 

90 

BRIDGE  A  BOOKLETS 

3 

3 

0 

BRIDGE  B  BOOKLETS 

2 

2 

2 

LANGUAGE  MINORITY  PROBE  BOOKLETS 

1 

1 

1 

EXCLUDED  STUDENT  QUESTIONNAIRES 

1 

1 

1 

TEACHER  QUESTIONNAIRES 

1 

1 

1 

SCHOOL  CHARACTERISTICS  QUESTIONNAIRES 

1 

1 

1 

COMPUTER  COORDINATOR  QUESTIONNAIRES 

1 

1 

1 

- -     TOTAL    -  - 

56 

72 

97 
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Table  15.2 


Number  of  Items  Administered 


COMMON  BACKGROUND 

READING 

BACKGROUND  AND  ATTITUDE 
COGNITIVE 

MATHEMATICS 

BACKGROUND  AND  ATTITUDE 
COGNITIVE 

SCIENCE 

BACKGROUND  AND  ATTITUDE 
COGNITIVE 

COMPUTER  COMPETENCE 

BACKGROUND  AND  ATTITUDE 
COGNITIVE 

U.S.  HISTORY 

BACKGROUND  AND  ATTITUDE 
COGNITIVE 

LITERATURE 

BACKGROUND  AND  ATTITUDE 
COGNITIVE 

EXCLUDED  STUDENTS 

TEACHER 
GENERAL 
ENGLISH 
MATHEMATICS 
SCIENCE 
U.S.  HISTORY 

SCHOOL  CHARACTERISTICS 

COMPUTER  COORDINATOR 

- -     TOTAL    -  - 


--Age  Class - 

9 

13 

17 

Total 

27 

28 

47 

57 

76 

90 

90 

113 

69 

73 

73 

119 

39 

119 

148 

195 

144 

311 

368 

509 

40 

85 

122 

152 

121 

191 

238 

436 

49 

62 

86 

114 

59 

136 

126 

212 

0 

0 

25 

25 

0 

0 

141 

141 

0 

0 

43 

43 

0 

0 

121 

121 

68 

68 

68 

68 

154 

150 

150 

173 

0 

13 

49 

49 

0 

60 

31 

71 

0 

85 

28 

96 

0 

0 

27 

27 

162 

166 

190 

262 

166 

166 

166 

166 

1174 

1803 

2337 

3149 
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Table  15.3 


Number  of  Cognitive  Items 
by  Type  of  Administration 


-Age  Class  

13  17 


READING 


MAIN  NAEP 

69 

73 

73 

BRIDGE  A 

31 

35 

0 

BRIDGE  B 

0 

0 

0 

MATHEMATICS 

MAIN  NAEP 

144 

311 

368 

BRIDGE  A 

68 

98 

0 

BRIDGE  B 

68 

98 

94 

SCIENCE 

MAIN  NAEP 

121 

191 

238 

BRIDGE  A 

63 

83 

0 

BRIDGE  B 

63 

83 

82 

COMPUTER  COMPETENCE 

MAIN  NAEP 

59 

136 

126 

BRIDGE  A 

0 

0 

0 

BRIDGE  B 

0 

0 

0 

U.S.  HISTORY 

MAIN  NAEP 

0 

0 

141 

BRIDGE  A 

0 

0 

0 

BRIDGE  B 

0 

0 

0 

LITERATURE 

MAIN  NAEP 

0 

0 

121 

BRIDGE  A 

0 

0 

0 

BRIDGE  B 

0 

0 

0 

ERIC 
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Table  15.4 


Characteristics  of  Schools  in 

Main  NAEP 

(Spiral) 

Sample 

 Age 

Class  — 

9 

13 

12 

Total 

TOTAL  ORIGINAL  SAMPLE 

by  / 

Id/. 

1  Q  Q  T 

COOPERATING 

594 

539 

409 

1542 

OUT -OF  RANGE  OR  CLOSED 

i  / 

1  Q 

NO  ELIGIBLES  ENROLLED 

9 

89 

22 

120 

DISTRICT  REFUSED 

46 

48 

45 

139 

SCHOOL  REFUSED 

Q1 
Ji 

51 

1  1  "7 

COOPERATION  RATE 

88.7 

88.1 

82.7 

86.8 

REPLACEMENT  FOR  REFUSALS 

41 

34 

37 

112 

COOPERATING 

38 

29 

24 

91 

TOTAL  COOPERATING  SCHOOLS 

632 

568 

433 

1633 

TOTAL  WITH  COMPLETED  QUESTIONNAIRES 

583 

521 

392 

1496 

REGION 

NORTHEAST 

142 

121 

79 

342 

SOUTHEAST 

139 

130 

101 

370 

CENTRAL 

165 

152 

121 

438 

WEST 

186 

165 

132 

483 

oUrlUULi  iirrj 

PUBLIC 

519 

461 

389 

1369 

PRIVATE 

46 

36 

23 

105 

CATHOLIC 

64 

68 

21 

153 

BIA 

2 

3 

0 

5 

DEPARTMENT  OF  DEFENSE 

1 

0 

0 

1 

SIZE  AND  TYPE  OF  COMMUNITY 

EXTREME  RURAL 

60 

57 

49 

166 

LOW  METROPOLITAN 

oi 

HIGH  METROPOLITAN 

67 

59 

42 

168 

MAIN  BIG  CITY 

64 

74 

42 

180 

URBAN  FRINGE 

76 

58 

47 

181 

MEDIUM  CITY 

111 

97 

70 

278 

SMALL  PLACE 

193 

169 

141 

503 

URBANICITY 

URBAN 

184 

171 

122 

477 

SUBURBAN 

252 

222 

171 

645 

RURAL 

196 

175 

140 

511 
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Table  15.4 
(continued) 


-Age  Class - 


GRADE  SPAN 

KINDERGARTEN  TO  GRADE  12 
KINDERGARTEN  TO  GRADE  6 
KINDERGARTEN  TO  GRADE  8 
GRADE  6  OR  7  TO  GRADE  8 
GRADE  7  TO  GRADE  9 
GRADE  7  TO  GRADE  12 
GRADE  9  TO  GRADE  12 
GRADE  10  TO  GRADE  12 
KINDERGARTEN  TO  GRADE  3 

NUMBER  OF  TEACHERS 
1  -  4 
5  .  9 
10  -  19 
20  -  49 
50  -  74 
75  -  99 
100+ 

NUMBER  OF  STUDENTS 
1  -  99 
100  -  299 
300  -  499 
500  -  749 
750  -  999 
1000  -  1499 
1500+ 

NO  INFORMATION 


i 

13 

17 

Total 

44 

42 

50 

136 

407 

107 

0 

514 

147 

159 

0 

306 

5 

163 

0 

168 

0 

59 

9 

68 

0 

35 

54 

89 

0 

3 

238 

241 

0 

0 

82 

82 

29 

0 

0 

29 

18 

14 

12 

44 

79 

60 

14 

153 

262 

143 

56 

461 

259 

274 

125 

658 

13 

56 

97 

166 

0 

17 

73 

90 

1 

4 

56 

61 

20 

14 

20 

54 

206 

143 

67 

416 

236 

144 

50 

430 

126 

139 

50 

^15 

28 

67 

49 

144 

12 

46 

86 

144 

3 

15 

111 

129 

1 

0 

0 

1 
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Table  15.5 

Number  of  Responses  to  Teacher  Questionnaire 


Teachers  with  Students  with 

Teachers          Students  in  Sample  Teachers  in  Sample 

Grade  3/Age  9 

ENGLISH                     774                         749  11222 


Grade  7/Age  13 

ENGLISH  270 

MATHEMATICS  263 

SCIENCE  251 


260  6466 
259  6746 
239  6896 


Grade  11/Age  17 

ENGLISH  315 

MATHEMATICS  362 

SCIENCE  307 

U.S.  HISTORY  259 


307  5905 

346  4266 

299  4365 

256  5401 


ERLC 


-308- 
9  O  ■ ' 

^  *-/'  /w 


Table  15.6 


Nvimber  of  Assessment  Sessions  by  Type  of  Administration 


MAIN  NAEP  (Spiral) 


BRIDGE  A 


BRIDGE  B 


 Age  Class  

9  13  17  Total 


REGULAR  1006  1008  1243  3257 

MAKE-UP  5  10  80  95 


REGULAR  484  291  775 

MAKE-UP  5  12  --  17 


REGULAR  222  223  199  644 

MAKE-UP  1  0  29  30 


TOTAL  1723  1544  1551  4818 
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Table  15.7 


Ntjmiber  of  Students  Assessed  and  Excluded 
by  Type  of  Administration 


ASSESSED 

MAIN  NAEP 
BRIDGE  A 
BRIDGE  B 


21287 
6932 
4042 


-Age  Class- 
13 


27668 
6200 
4178 


17 


39753 


3868 


Total 

88708 
13132 
12088 


EXCLUDED 

MAIN  AND  BRIDGE  B 
BRIDGE  A 


1133 
343 


1382 
386 


1965 


4480 

729 


TOTAL  33737  39814  45586  119137 


-310-    2^  ' 


Table  15.8 


Ntimber  of  Students  in  the  Main  NAEP  (Spiral)  Sample 

Grade  3/Age  9 


TOTAL 

SEX: 
MALE 
FEMALE 

RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 

REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 

PARENTS  ED: 

LESS  THAN  HIGH  SCHOOL 
HIGH  SCHOOL 

GREATER  THAN  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 

SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


Age 
1663? 


8422 
8210 


10323 
2966 
2696 
647 


3201 
4115 
3935 
5672 


712 
2131 

876 
5124 
7491 


1072 
2093 
2008 
2181 
1888 
3106 
4284 


Grade 


18033 


9124 
8909 


10896 
3356 
3123 
658 


3396 
4647 
4152 
6103 


809 
2292 

9J1 
5565 
8143 


1137 
2343 
2138 
2363 
1994 
3397 
4661 


-Eligible  by- 

Age  6e  Grade 


13378 


6543 
6835 


8585 
2287 
2033 
473 


2544 
3347 
3212 
4462 


522 
1670 

707 
4316 
5972 


866 
1545 
1666 
1748 
1520 
2502 
3531 


Age  or  Grade 


21287 


11003 
10284 


12634 
4035 
3786 
832 


4053 
5415 
4875 
7313 


999 

2753 
1120 
6373 
9662 


1343 
2891 
2480 
2796 
2362 
4001 
5414 
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Table  15.9 


Nvraiber  of  Students  in  the  Main  NAEP  (Spiral)  Sample 
Grade  7/Age  13 


Ape 


Grade 


-Eligible  by- 
Age  &  Grade 


Age  or  Grade 


TOTAL 


SEX: 


MALE 
FEMALE 


20554 


10232 
10322 


23527 


11986 
11541 


16413 


7923 
8490 


27668 


14295 
13373 


RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 


12460 
4018 
3339 
737 


13871 
4846 
3966 
844 


10263 
3079 
2506 
565 


16068 
5785 
4799 
1016 


REGION : 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


3451 
5658 
5154 
6502 


4004 
6545 
5991 
7254 


2634 
4583 
4315 
5054 


4821 
7620 
6830 
8702 


PARENTS  ED: 

LESS  THAN  HIGH  SCHOOL 
HIGH  SCHOOL 

GREATER  THAN  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 


1556 
5661 
2962 
7410 
2733 


1998 
6549 
3314 
8194 
3176 


1151 
4502 
2439 
6158 
1971 


2403 
7708 
3837 
9446 
3938 


SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


1017 
2004 
1959 
3146 
2885 
3414 
6129 


1161 
2632 
2055 
3619 
3324 
3980 
6756 


819 
1542 
1497 
2526 
2315 
2812 
4902 


1359 
3094 
2517 
4239 
3894 
4582 
7983 
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Table  15.10 


Number  of  Students  in  the  Main  NAEP  (Spiral)  Sample 
Grade  11/Age  17 


TOTAL 

SEX: 
MALE 
FEMALE 

RACE: 
WHITE 
BIACK 
HISPANIC 
OTHER 

REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 

PARENTS  ED: 

LESS  THAN  HIGH  SCHOOL 
HIGH  SCHOOL 

GREATER  THAN  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 

SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


31782 


15828 
15954 


22204 
5360 
3363 
855 


6331 
8122 
8298 
9163 


3050 
9004 
6897 
11496 
1189 


1337 
2336 
3600 
3567 
4331 
6258 
10353 


 Eligible  by- 

Grade      Age  &  Grade 


31938 


15797 
16141 


22603 
5230 
3170 
935 


6599 
7988 
8215 
9270 


2944 
8847 
7021 
11865 
1112 


1349 
2083 
3791 
3348 
4394 
6351 
10622 


23967 


11452 
12515 


17862 
3401 
2103 
601 


4678 
5931 
6556 
6887 


1916 
6585 
5473 
9228 
668 


991 
1300 
2920 
2366 
3369 
4922 
8099 


Age  or  Grade 
39753 


20173 
19580 


26945 
7189 
4430 
1189 


8252 
10179 

9957 
11546 


4078 
11266 

8445 
14133 

1633 


1695 
3119 
4471 
4549 
5356 
7687 
12876 
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Table  15.11 
Number  of  Students  in  the  Bridge  A  Sample 


TOTAL 

SEX: 
MALE 
FEMALE 

RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 

REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 

PARENTS  ED: 

LESS  THAN  HIGH  SCHOOL 
HIGH  SCHOOL 

GREATER  THAN  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 

SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


Age  9  Age  13         Age  17 

6932  6200  0 

3487  3102  0 

3445  3096  0 

4534  3667  0 

851  1462  0 

1022  868  0 

525  203  0 

1708  1485  0 

1583  1418  0 

1714  1613  0 

1937  1788  0 

292  502  0 

1057  1795  0 

466  923  0 

2659  2311  0 

2445  561  0 

260  232  0 

513  886  0 

1043  692  0 

693  788  0 

1016  580  0 

1249  1318  0 

2158  1704  0 
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Table  15.12 
Number  of  Students  in  the  Bridge  B  Sample 


TOTAL 

SEX: 
MALE 
FEMALE 

MCE: 
WHITE 
BLACK 
HISPANIC 
OTHER 

REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 

PARENTS  ED: 

LESS  THAN  HIGH  SCHOOL 
HIGH  SCHOOL 

GREATER  THAN  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 

SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


Age  9  Age  13  Age  17 

4042  4178  3868 

2014  2063  1870 

2028  2115  1998 

2539  2628  2670 

648  670  630 

682  695  429 

173  185  139 

812  772  746 

999  1120  1081 

943  956  910 

1316  1346  1145 

209  342  382 

586  1145  1074 

229  612  902 

1331  1588  1378 

1656  471  116 

287  250  145 

437  364  308 

400  345  410 

411  582  427 

489  513  598 

821  684  739 

1197  1440  1241 
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Table  15.13 

Excluded  Student  Sample  by  Demographic  Characteristics 

Grade  3/Age  9 


Bridge  A   Main  NAEP  and  Bridge  B  

Age  Age      Grade    Age  and  Grade    Age  or  Grade 


TOTAL  343  802  767  436  1133 
SEX: 

MALE  229  498  465  250  713 

FEMALE  114  304  302  186  420 

RACE: 

WHITE  172  284  255  137  402 

BLACK  59  152  133  64  221 

HISPANIC  92  305  323  201  427 

OTHER  20  61  56  34  83 

REGION: 

NORTHEAST  80  136  120  80  176 

SOUTHEAST  79  132  127  53  206 

CENTRAL  54  104  94  39  159 

WEST  130  430  426  264  592 
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Table  15.14 


Excluded  Student  Sample  by  Demographic  Characteristics 

Grade  7/Age  13 


Bridge  A 
Age 


TOTAL  386. 
SEX: 

MALE  243 

FEMALE  143 

RACE: 

WHITE  126 

BLACK  96 

HISPANIC  122 

OTHER  42 

REGION: 

NORTHEAST  138 

SOUTHEAST  67 

CENTRAL  66 

WEST  115 


 Main  NAEP  and  Bridge  B  

Age      Grade    Age  and  Grade    Age  or  Grade 


773  1052  443  1382 

495  686  277  904 

278  366  166  478 

294  452  156  590 

193  271  116  348 

230  267  141  356 

56  62  30  88 

77  125  47  155 

181  307  122  366 

143  144  49  238 

372  476  225  623 


Table  15.15 


Excluded  Student  Sample  by  Demographic  Characteristics 

Grade  11/Age  17 


Bridge  A 
Age 


TOTAL  0 
SEX: 

MALE  0 

FEMALE  0 

RACE: 

WHITE  0 

BLACK  0 

HISPANIC  0 

OTHER  0 

REGION : 

NORTHEAST  0 

SOUTHEAST  0 

CENTRAL  0 

WEST  0 


 Main  NAEP  and  Bridge  B  

Age      Grade    Age  and  Grade    Age  or  Grade 


1368  1150  553  1965 

827  718  330  1215 

530  416  213  733 

588  510  245  853 

404  329  162  571 

281  210  110  381 

95  101  36  160 

222  189  100  311 

342  271  130  483 

288  261  109  440 

516  429  214  731 
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Table  15.16 


Number  of  Students  by  Grade/Age 
and  T3rpe  of  Assessment 


ASSESSMENT 
TYPE 


GRADE  3/AGE  9 

Sum  of 
Total  Weights 


GRADE  7/AGE  13 

Sum  of 
Total  Weights 


GRADE  11/AGE  17 
SuiL  of 
Total  Weights 


Spiral  ^ 
Bridge  B- -Booklet  4^ 
Bridge  B- -Booklet  5 
Excluded  Students- - 
Combined  Spiral  and 
Bridge  B  Samples 


21287  3931992 
1994  3151352 
2048  3121844 


1133  137280 


27668  4007907 
2032  3008026 
2146  3028806 


1382  143847 


39753  4136965 
1934  3240017 
1934  3252949 


1965  120169 


Bridge  A 
Bridge  A 
Bridge  A 


-Booklet 
-Booklet 


-k-k 


Booklet  3 
Excluded  Students- - 
Bridge  A  Samples 


2315 
2361 
2256 

343 


3098639 
3104555 
3112834 

131664 


2075 
2054 
2071 

386 


2937402 
2950983 
2943837 

148916 


^Sample  for  both  age  and  grade  using  new  age  definitions 
^^ample  for  age  only  using  new  age  definitions 
Sample  for  age  only  using  old  age  definitions 


-319- 

ERLC 


Table  15.17 


Number  of  Spiral  Students,  Grade  3/Age  9 
(Booklets  6  -  51) 


GRADE  <  3 


<  9 


-AGE- 
-  9 


>  9 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


2342 
432377 
19486 
4.51 


0 
0 


2342 
432377 
19486 
4.51 


GRADE  -  3 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1236 
208510 
15176 
7.28 


13378 
2530844 
9749 
0.39 


3419 
581923 
15623 
2.68 


18033 
3321278 
14611 
0.44 


GRADE  >  3 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


912 
178337 
18177 
10.19 


0 
0 


912 
178337 
18177 
10.19 


GRADE  TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1236 
208510 
15176 
7.28 


16632 
3141559 
13874 
0.44 


3419 
581923 
15623 
2.68 


21287 
3931992 
18966 
0.48 
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Table  15.18 


Number  of  Spiral  Students,  Grade  7/Age  13 
(Booklets  6  -  67) 


GRADE  <  7 


<  13 


-AGE-- 
-  13 


>  13 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


2775 
646504 
23559 
3.64 


0 
0 


2775 
646504 
23559 
3.64 


GRADE 


UNWEIGHTED  N  1751 

WEIGHTED  N  257520 

STANDARD  ERROR  16768 

COEFF.  OF  VAR.  6.51 

GRADE  >  7 


16413  5363  23527 

2062373  759124  3079017 

3873  16735  10959 

0.19  2.20  0.36 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  TOTAL 


0  1366 
0  282386 
22763 
8.06 


0  1366 
0  282386 
22763 
8.06 


UNWEIGHTED  N  1751 

WEIGHTED  N  257520 

STANDARD  ERROR  16768 

COEFF.  OF  VAR.  6.51 


20554  5363  27668 

2991263  759124  4007907 

12922  16735  16317 

0.43  2.20  0.41 
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Table  15.19 


Number  of  Spiral  Students,  Grade  11/Age  17 
(Booklets  6  -  95) 


GRADE  <  11 


<  17 


-AGE-. 
-  17 


>  17 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


5613 
692940 
16672 
2.41 


0 
0 


5613 
692940 
16672 
2.41 


GRADE  -  11 

UNWEIGHTED  N  3264  23967  4707  31938 

WEIGHTED  N  356364  2270022  561849  3188235 

STANDARD  ERROR  17845  3696  16124  8536 

COEFF.  OF  VAR.  5.01  0.16  2.87  0.27 

GRADE  >  11 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


2202 
255790 
17040 
6.66 


0 
0 


2202 
255790 
17040 
6.66 


GRADE  TOTAL 


UNWEIGHTED  N  3264  31782  4707  39753 

WEIGHTED  N  356364  3218752  561849  4136965 

STANDARD  ERROR  17845  8501  16124  13025 

COEFF.  OF  VAR.  5.01  0.26  2.87  0.31 
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Table  15.20 


Number  of  Bridge  Students,  Age  9 
(Booklets  i  -  5) 


<  MODAL  GRADE 


1* 


2* 


•Bridge  Booklet - 
3* 


4** 


5** 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  EEIROR 
COEFF.  OF  VAR. 

-  MODAL  G.ADE 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  EEIROR 
COEFF.  OF  VAR. 

>  MODAL  GRADE 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


825 

loe^^.ns 

55927 
5.25 


1482 
2023333 
54618 
2.70 


8 

10988 
3597 
33.64 


795 
1013447 
53245 
5.25 


1559 
2083592 
63036 
3.03 


7 

7515 
2793 
37.16 


758 
1084801 
62079 
5.72 


1491 
2020543 
62539 
3.10 


7 

7489 
2521 
33.66 


280 
385478 
33886 
8.79 


1610 
2604518 
48797 
1.87 


104 
161557 
56003 
34.66 


280 
430998 
36675 
8.51 


1667 
2554347 
41097 
1.61 


101 
136499 
17635 
12.92 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


2315 
3098639 
16593 
0.54 


2361 
3104555 
20282 
0.65 


2256 
3112834 
14390 
0.46 


1994 
3151352 
20051 
0.64 


2048 
3121844 
26874 
0.86 


♦Students  who  took  these  booklets  have  reading,  mathematics,  and  science 
plausible  values;  modal  grade  is  grade  4. 

**Students  who  took  these  booklets  have  mathematics  and  science  plausible 
values ;  modal  grade  is  grade  3 . 


ER.IC 
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Table  15.21 


Number  of  Bridge  Students,  Age  13 
(Booklets  1  -  5) 


<  MODAL  GRADE 


MODAL  GRADE 


>  MODAL  GRADE 


TOTAL 


1* 


2* 


•Bridge  Booklet - 
3* 


4** 


5** 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


693 
1000038 
74097 
7.41 


651 
975363 
68626 
7.04 


655 
915681 
49935 
5.45 


248 
409390 
74647 
18.23 


260 
452561 
137092 
30.29 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1372 
1928568 
77648 
4.03 


1390 
1958298 
79717 
4.07 


1403 
2014274 
51542 
2.56 


1642 
2404636 
93872 
3.90 


1743 
2395404 
126803 
5.29 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


10 
8796 
6317 
71.82 


13 
17322 
6447 
37.22 


13 
13882 
7946 
57.24 


142 
194000 
32980 
17.00 


143 
180841 
22955 
12.69 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


2075 
2937402 
21332 
0.73 


2054 
2950983 
24449 
0.83 


2071 
2943837 
21182 
0.72 


2032 
3008026 
22738 
0.76 


2146 
3028806 
15282 
0.50 


♦Students  who  took  these  booklets  have  reading,  mathematics,  and  science 
plausible  values;  modal  grade  is  grade  8. 

**Students  who  took  these  booklets  have  mathematics  and  science  plausible 
values ;  modal  grade  is  grade  7 . 
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Table  15.22 


Niunber  of  Bridge  Students,  Age  17 
(Booklets  4-5) 


-Bridge  Booklet - 
3 


4* 


5* 


<  MODAL  GRADE 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


0 
0 


0 
0 


358 
521912 
47386 
9.08 


332 
569292 
47724 
8.38 


-  MODAL  GRADE 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


0 
0 


0 
0 


1415 
2406593 
52550 
2.18 


1473 
2484713 
52069 
2.10 


>  MODAL  GRADE 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


0 
0 


0 
0 


161 
311511 
32298 
10.37 


129 
198945 
21398 
10.76 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


0 
0 


0 
0 


1934 
3240017 
13772 
0.43 


1934 
3252949 
10159 
0.31 


♦Students  who  took  these  booklets  have  mathematics  and  science  plausible 
values;  modal  grade  is  grade  11. 


-325- 


3/n 


ERIC 


Table  15.23 


Estimated  Total  Number  of  Students  in  the  Population 
Eligible  for  Spiral  Assessment 

Grade  3/Age  9,  Weighted 


TOTAL 


Age 

3141559 


-Eligible  by- 
Grade 

3321278 


Grade/Age 
3931992 


SEX: 
MALE 
FEMALE 


1567321 
1574237 


1658242 
1663035 


2010984 
1921008 


RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 


2271105 
441950 
316905 
111598 


2377130 
475627 
352624 
115897 


2773153 
580336 
436379 
142124 


REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


657236 
699613 
923216 
908020 


681455 
752156 
965033 
963708 


817351 
890668 
1137030 
1146791 


PARENTS  ED: 

<  HIGH  SCHOOL 
HIGH  SCHOOL 
>  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 


131316 
416636 
166727 
968207 
1411270 


141487 
432832 
177790 
1034473 
1492215 


176436 
520793 
208244 
1186181 
1778771 


SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


293166 
258954 
381331 
297740 
364431 
535302 
1010635 


302728 
282235 
405166 
311031 
375428 
565747 
1078942 


359727 
352024 
466861 
373471 
447383 
675622 
1256904 
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Table  15.24 


Estimated  Total  Number  of  Students  in  the  Population 
Eligible  for  Spiral  Assessment 

Grade  7/Age  13,  Weighted 


TOTAL 

SEX: 

MALE 
FEMALE 

RACE: 

WHITE 
BLACK 
HISPANIC 
OTHER 

REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 

PARENTS  ED: 

<  HIGH  SCHOOL 
HIGH  SCHOOL 
>  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 


Age 
2991263 


1541750 
1449513 


2145908 
436915 
296344 
112096 


610242 
687441 
816427 
898675 


217746 
821310 
444494 
1100343 
382989 


•Eligible  by- 
Grade 

3079017 


1593965 
1485052 


2178573 
460779 
323566 
116099 


638455 
705074 
849132 
908956 


245840 
867910 
454059 
1109545 
374855 


Grade/Age 
4007907 


2128490 
1879417 


2781946 
636841 
431632 
157488 


830292 
936310 
1085672 
1187272 


334210 
1122312 

573892 
1402384 

539028 


SIZE  AND  TYPE  OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


184026 
212154 
335761 
342193 
355191 
451977 
1109961 


198493 
250926 
326645 
361595 
366450 
472968 
1101941 


249876 
328053 
435215 
459189 
478791 
604822 
1451962 
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Table  15.25 


Estimated  Total  Number  of  Students  in  the  Population 
Eligible  for  Spiral  Assessment 

Grade  11/Age  17,  Weighted 


TOTAL 


Age 
3218752 


-Eligible  by- 
Grade 

3188235 


Grade/ Age 
4136965 


SEX: 


MALE 
FEMALE 


1637504 
1581248 


1605264 
1582971 


2146872 
1990093 


RAGE: 
WHITE 
BLACK 
HISPANIC 
OTHER 


2454103 
425180 
239970 
99499 


2429584 
420955 
231839 
105858 


3053113 
599499 
342771 
141581 


REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


760527 
679527 
936492 
857844 


772512 
660859 
917117 
852943 


997946 
897434 
1159932 
1103710 


PARENTS  ED: 

<  HIGH  SCHOOL 
HIGH  SCHOOL 
>  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 


286613 
901963 
686664 
1214724 
111782 


266d33 
867463 
695486 
1237960 
103724 


394278 
1162109 

865573 
1532637 

158414 


SIZE  AND  TYPE  OF  COMl^'NITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


140537 
192108 
404668 
273140 
496195 
504865 
1207238 


139899 
168539 
437120 
255547 
496725 
503134 
1187271 


178611 
271577 
527094 
363232 
631919 
638491 
1526041 
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Table  15.26 


Estimated  Total  Number  of  Students 
Eligible  for  Assessment  by  Bridge  Sample 

Age  9,  Weighted 


BOOKLET  1 


BRIDGE  A  - 
BOOKLET  2 


BOOKLET  3 


  BRIDGE  B   

BOOKLET  4      BOOKLET  5 


TOTAL  3098639  3104555 
SEX: 

MALE  1513328  1528517 

FEMALE  1585312  1576038 

RACE: 

WHITE  2239076  2245501 

BLACK  427691  432972 

HISPANIC  321255  309898 

OTHER  110617  116184 

REGION: 

NORTHEAST  647611  673694 

SOUTHEAST  707711  683262 

CENTRAL  894822  897240 

WEST  851126  851944 

PARENTS  ED: 

<  HIGH  SCHOOL  129664  133526 

-  HIGH  SCHOOL  480533  540712 

>  HIGH  SCHOOL  199661  206521 

GRADUATED  COLLEGE        1180756  1200317 

UNKNOWN  1104628  1021203 

SIZE  AND  TYPE 

OF  COMMUNITY: 

RURAL  131748  157369 

DISADVANTAGED  URBAN      183010  177983 

ADVANTAGED  URBAN  517107  513362 

BIG  CITY  251498  272930 

FRINGE  410427  424339 

MEDIUM  482355  455960 

SMALL  1122495  1102612 


3112834 


1614911 
1497923 


2251748 
429194 
315744 
116147 


646732 
708425 
884580 
880282 


130625 
502159 
208628 
1139467 
1124254 


153949 
194386 
534404 
266679 
384390 
426875 
1152150 


3151352 


1586069 
1565283 


2281491 
444069 
312820 
112972 


700123 
667800 
992571 
836375 


148026 
407702 
202829 
1060530 
1280633 


362786 
317002 
360928 
316439 
337960 
587028 
869208 


3121844 


1558197 
1563647 


2246936 
438022 
323488 
113397 


706970 
704131 
879846 
837373 


172975 
475574 
143644 
1075184 
1247992 


193753 
232967 
372294 
172480 
603800 
441577 
1104973 
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Table  15.27 


Estimated  Total  Number  of  Students 
Eligible  for  Assessment  by  Bridge  Sample 

Age  13,  Weighted 


  BRIDGE  A   

BOOKLET  1      BOOKLET  2      BOOKLET  3 


  BRIDGE  B   

BOOKLET  4      BOOKLET  5 


TOTAL 


2937402 


2950983 


2943837 


3008026 


3028806 


SEX: 
MALE 
FEMALE 


1467700 
1469456 


1465382 
1485600 


1464555 
1479088 


1496077 
1511950 


1518031 
1510774 


RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 


2126752 
430424 
287880 
92346 


2141182 
428826 
271851 
109124 


2125403 
440654 
276136 
101645 


2154960 
441065 
292548 
119453 


2169537 
449180 
299453 
110635 


REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


653213 
714405 
782547 
828271 


663776 
736100 
766492 
824659 


661239 
735176 
767112 
820122 


597610 
697252 
844402 
871474 


582830 
745166 
851214 
859251 


PARENTS  ED: 

<  HIGH  SCHOOL  247165  233860  209186 

-  HIGH  SCHOOL  866426  887299  943903 

>  HIGH  SCHOOL  493495  435220  427030 

GRADUATED  COLLEGE  1044757  1136582  1085363 

UNKNOWN  243633  216447  236035 


208139 
803110 
472907 
1203210 
316897 


225451 
855055 
426546 
1178812 
328894 


SIZE  AND  TYPE 
OF  COMMUNITY: 

RURAL  181662  162410  186124  192069  218660 

DISADVANTAGED  URBAN      273163  232200  278075  227973  176697 

ADVANTAGED  URBAN  336985  340383  336147  192679  357848 

BIG  CITY  343696  327501  337060  479026  365954 

FRINGE  252105  258556  255761  384401  373994 

MEDIUM  466608  482638  477884  374746  397159 

SMALL  1083183  1147294  1072786  1157132  1138494 
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Table  15.28 


Estimated  Total  Number  of  Students 
Eligible  for  Assessment  by  Bridge  Sample 

Age  17,  Weighted 


BOOKLET  1 


BRIDGE  A  - 
BOOKLET  2 


BOOKLET  3 


  BRIDGE  B   

BOOKLET  4      BOOKLET  5 


TOTAL 

SEX: 
MALE 
FEMALE 

RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 

REGION : 
NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 

PARENTS  ED: 

<  HIGH  SCHOOL 
-  HIGH  SCHOOL 
>  HIGH  SCHOOL 
GRADUATED  COLLEGE 
UNKNOWN 

SIZE  AND  TYPE 
OF  COMMUNITY: 
RURAL 

DISADVANTAGED  URBAN 

ADVANTAGED  URBAN 

BIG  CITY 

FRINGE 

MEDIUM 

SMALL 


3240017  3252949 


0 
0 


0 
0 
0 
0 


0 
0 
0 
0 


0 
0 
0 
0 
0 


0 
0 
0 
0 
0 
0 
0 


0 
0 


0 
0 
0 
0 


0 
0 
0 
0 


0 
0 
0 
0 
0 


0 
0 
0 
0 
0 
0 
0 


0 
0 


0 
0 
0 
0 


0 
0 
0 
0 


0 
0 
0 
0 
0 


0 
0 
0 
0 
0 
0 
0 


1552203 
1687814 


2465764 
430391 
241481 
102381 


799647 
783775 
843631 
824428 


271395 
882974 
810293 
1165164 
96000 


43064 
280426 
348843 
253271 
566581 
647123 
1100710 


1626960 
1625989 


2468639 
429952 
248189 
106169 


737680 
700061 
983235 
847085 


262663 
920148 
748687 
1218276 
88064 


158648 
125606 
472761 
321808 
601024 
448500 
1124602 
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Table  15.29 


Estimated  Total  Population  of  Excluded  Students 
Grade  3/Age  9,  Weighted 


TOTAL 


Bridge  A 
Age 

131664 


 Main  and  Bridge  B  

Age         Grade      Age  or  Grade 


89319 


94135 


137280 


SEX: 


MALE 
FEMALE 


86979 
44685 


58325 
30994 


59065 
35070 


89291 
47989 


RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 


66692 
21760 
33896 
9316 


40510 
15508 
23836 
9464 


41773 
15614 
26445 
10303 


62623 
24279 
36280 
14097 


REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


28164 
29640 
22779 
51081 


19264 
15469 
15823 
38762 


18377 
17669 
17440 
40649 


25983 
26944 
26875 
57477 
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Table  15.30 


Estimated  Total  Population  of  Excluded  Students 
Grade  7/Age  13,  Weighted 


TOTAL 

SEX: 
MALE 
FEMALE 

RACE: 
WHITE 
BLACK 
HISPANIC 
OTHER 

REGION: 

NORTHEAST 
SOUTHEAST 
CENTRAL 
WEST 


Bridge  A 
Age 

148916 


9C477 
52440 


57522 
34253 
41704 
15438 


53251 
28983 
26243 
40440 


Age 
77668 


49596 
28072 


35017 
16896 
16827 
8928 


7374 
19519 
17326 
33449 


-Main  and  Bridge  B  

Grade      Age  or  Grade 


105840 


68928 
36912 


53700 
22533 
20431 
176 


13202 
31800 
18437 
42401 


143847 


94201 
49646 


72270 
30430 
27444 
13704 


16685 
39609 
29979 
57574 
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Table  15.31 


Estimated  Total  Population  of  Excluded  Students 

Grade  11/Age  17 ,  Weighted 

Bridge  A   Main  and  Bridge  B  

Age  Age          Grade  Age  or  Grade 

TOTAL  0  79149        72744  120169 

SEX: 

MALE  0  48103        45950  74512 

FEMALE  0  30659        26208  45039 

RACE: 

WHITE  0  37956        35561  57498 

BLACK  0  19937        17309  29513 

HISPANIC  0  13786          9655  18292 

OTHER  0  7469        10219  14867 

REGION: 

NORTHEAST  0  12914        12638  19656 

SOUTHEAST  0  17536        15369  26199 

CENTRAL  0  19676        19391  31156 

WEST  0  29024        25346  43157 
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Table  15.32 


Spiral  Sample  Students  Who  Have  Reading  Scale  Scores 
(Variables  REDVALl  to  REDVAL5) 


Grade  3/Age  9 


GRADE  <  3 


<  9 


-AGE- 
-  9 


>  9 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.   OF  VAR. 


0 
0 


1281 
238526 
12042 
5.05 


0 
0 


1281 
238526 
12042 
5.05 


GRADE  -  3 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  3 


663  7289 

111321  1386813 

9060  8288 

8.14  0.60 


1841  9793 

314145  1812279 

9237  9488 

2.94  0.52 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


0  501 
0  99616 
11672 
11.72 


0  501 
0  99616 
11672 
11.72 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


663  9071 

111321  1724955 

9060  9982 

8.14  0.58 


1841  11575 

314145  2150421 

9237  11533 

2 . 94  0 .  54 
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Table  15.33 


Spiral  Sample  Students  Who  Have  Reading  Scale  Scores 
(Variables  REDVALl  to  REDVAL5) 

Grade  7/Age  13 


GRADE  <  7 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.   OF  VAR. 


<  13 


0 
0 


-AGE-- 
-  13 


1130 
270315 
12485 
4.62 


>  13 


0 
0 


TOTAL 


1130 
270315 
12485 
4.62 


GRADE  -  7 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


713 
107052 
7977 
7.45 


6638 
835432 
6210 
0.74 


2162 
303535 
9439 
3.11 


9513 
1246019 
6893 
0.55 


GRADE  >  7 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


528 
111168 
9560 
8.60 


0 
0 


528 
111168 
9560 
8.60 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


713 
107052 
7977 
7.45 


8296 
1216915 
11441 
0.94 


2162 
303535 
9439 
3.11 


11171 
1627503 
11292 
0.69 
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Table  15.34 


Spiral  Sample  Students  Who  Have  Reading  Scale  Scores 
(Variables  REDVALl  to  REDVAL5) 

Grade  11/Age  17 


GRADE  <  11 


<  17 


■AGE- • 
-  17 


>  17 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  -  11 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  11 


0 
0 


1703 
187073 
10196 
5.45 


2889 
361108 
9755 
2.70 


12393 
1165375 
6762 
0.58 


0 
0 


2414 
285703 
9868 
3.45 


2889 
361108 
9755 
2.70 


16510 
1638151 
7343 
0.45 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


1136 
130446 
8579 
6.58 


0 
0 


1136 
130446 
8579 
6.58 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1703 
187073 
10196 
5.45 


16418 
1656930 
76^3 
0.46 


2414 
285703 
9868 
3.45 


20535 
2129706 
8340 
0.39 
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Table  15.35 


Spiral  Sample  Students  Who  Have  Mathematics  Scale  Scores 
(Variables  MRPCMPl  to  MRPCMP5) 

Grade  3/Age  9 


GRADE  <  3 


<  9 


-AGE- 
-  9 


>  9 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  -  3 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  3 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


0 
0 


769 
132771 
10248 
7.72 


0 
0 


1462 
271299 
15134 
5.58 


8102 
1530675 
10781 
0.70 


527 
102306 
10523 
10.29 


0 
0 


2u74 
354866 
10279 
2.90 


0 
0 


1462 
271299 
15134 
5.58 


10945 
2018312 
15656 
0.78 


527 
102306 
10523 
10.29 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


769 
132771 
10248 
7.72 


10091 
1904280 
14811 
0.78 


2074 
354866 
10279 
2.90 


12934 
2391917 
18054 
0.75 
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Table  15.36 


Spiral  Sample  Students  Who  Have  Mathematics  Scale  Scores 
(Variables  MRPCMPl  to  MRPCMP5) 

Grade  7/Age  13 


GRADE  <  7 


<  13 


-AGE-- 
-  13 


>  13 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  EEIROR 
COEFF.  OF  VAR. 

GRADE  -  7 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  7 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


0 
0 


911 
134283 
10060 
7.49 


0 
0 


1448 
333080 
15348 
4.61 


8462 
1063563 
8324 
0.78 


692 
142784 
12783 
8.95 


0 
0 


2812 
398199 
8856 
2.22 


0 
0 


1448 
333080 
15348 
4.61 


12185 
1596045 
10692 
0.67 


692 
142784 
12783 
8.95 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


911 
134283 
10060 
7.49 


10602 
1539427 
12874 
0.84 


2812 
398199 
8856 
2.22 


14325 
2071909 
14557 
0.70 
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Table  15.37 


Spiral  Sample  Students  Who  Have  Mathematics  Scale  Scores 
(Variables  MRPCMPl  to  MRPCMP5) 

Grade  11/Age  17 


GRADE  <  11 


<  17 


•AGE-- 
-  17 


>  17 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


2011 
245638 
8513 
3.47 


0 
0 


2011 
245638 
8513 
3.47 


GRADE  -  11 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1157 
125937 
7688 
6.10 


8850 
842302 
6889 
0.82 


1843 
222496 
8002 
3.60 


11850 
1190734 
9237 
0.78 


GRADE  >  11 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.   OF  VAR. 


0 
0 


799 
94768 
7425 
7.84 


0 
0 


799 
94768 
7425 
7.84 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1157 
125937 
7688 
6.10 


11660 
1182708 
6612 
0.56 


1843 
222496 
8002 
3.60 


14660 
1531141 
9626 
0.63 


V 
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Table  15.38 


Spiral  Sample  Students  Who  Have  Science  Scale  Scores 
(Variables  SRPCMPl  to  SRPCMP5) 

Grade  3 /Age  9 


<  9 


GRADE  <  3 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  -  3 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  3 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


0 
0 


743 
124264 
9532 
7.67 


0 
0 


-AGE- 
-  9 


1420 
264118 
13164 
4.98 


8140 
1535163 
11748 
0.77 


520 
105268 
11319 
10.75 


>  9 


0 
0 


2163 
368634 
10176 
2.76 


0 
0 


TOTAL 


1420 
264118 
13164 
4.98 


■'.1046 
2028061 
13289 
0.66 


520 
105268 
11319 
10.75 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


743 
124264 
9532 
7.67 


10080 
1904549 
11629 
0.61 


2163 
368634 
10176 
2.76 


12986 
2397447 
14036 
0.59 
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Table  15.39 


Spiral  Sample  Students  Who  Have  Science  Scale  Scores 
(Variables  SRPCMPl  to  SRPCMP5) 

Grade  7/Age  13 


GRADE  <  7 


<  13 


-AGE-- 
-  13 


>  13 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


1442 
327397 
14965 
4.57 


0 
0 


1442 
327397 
14965 
4.57 


GRADE 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


923 
133754 
9247 
6.91 


8473 
1068670 
8333 
0.78 


2746 
388401 
11362 
2.93 


12142 
1590825 
11054 
0.69 


GRADE  >  7 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


0 
0 


692 
145400 
13434 
9.24 


0 
0 


692 
145400 
13434 
9.24 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


923 
133754 
9247 
6.91 


10607 
1541468 
12429 
0.81 


2746 
388401 
11362 
2.93 


14276 
2063623 
15009 
0.73 
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Table  15.40 


Spiral  Sample  Students  Who  Have  Science  Scale  Scores 
(Variables  SRPCMPl  to  SRPCMP5) 

Grade  11/Age  17 


GRADE  <  11 


<  17 


-AGE-- 
-  17 


>  17 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  -  11 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  11 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


0 
0 


1172 
127558 
6689 
5.24 


0 
0 


2079 
254934 
7272 
2.85 


8878 
845271 
6818 
0.81 


851 
101266 
7975 
7.88 


0 
0 


1694 
201564 
6903 
3.42 


0 
0 


2079 
254934 
7272 
2.85 


11744 
1174394 
8186 
0.70 


851 
101266 
7975 
7.88 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


1172 
127558 
6689 
5.24 


11808 
1201472 
9570 
0.80 


1694 
201564 
6903 
3.42 


14674 
1530594 
10636 
0.69 
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Table  15.41 


Spiral  Sample  Students  Who  Have  History  Scale  Scores 
(Variable  HISVAL) 

Grade  11/Age  17 


GRADE  <  11 


 AGE  

<  17  -  17  >  17  TOTAL 


UNWEIGHTED  N  0  0  0  0 

WEIGHTED  N  0  0  0  0 

STANDARD  ERROR  -  -  -  - 

COEFF.  OF  VAR.  -  -  -  - 


vrRADE  «  11 


UNWEIGHTED  N 
WEIGHTED  N 
STAN  'ARD  ERROR 
CCEFF    OF  VAR, 


819  5862 

87824  548766 

4649  5386 

5,29  0.98 


1131  7812 

135695  772285 

5106  7030 

3,76  0,91 


GRADE  >  11 

UNWEIGHTED  N  0  0  0  0 

WEIGHTED  N  0  0  0  0 

STANDARD  ERROR  -  -  -  - 

COEFF.  OF  VAR.  -  -  -  - 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR, 


819  5862 

87824  548766 

4649  5386 

5,29  0.98 


1131  7812 

135695  772285 

5106  7030 

3,76  0.91 
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Table  15.42 


Spiral  Sample  Students  Who  Have  Literature  Scale  Scores 

(Variable  LITVAL) 

Grade  11/Age  17 


GRADE  <  11 


<  17 


-AGE-- 
-  17 


>  17 


TOTAL 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  -  11 

UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

GRADE  >  11 


0 
0 


819 
87824 
4649 
5.29 


0 
0 


5862 
548766 
5386 
0.98 


0 
0 


1131 
135695 
5106 
3.76 


0 
0 


7812 
772285 
7030 
0.91 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 

TOTAL 


0 
0 


0 
0 


0 
0 


UNWEIGHTED  N 
WEIGHTED  N 
STANDARD  ERROR 
COEFF.  OF  VAR. 


819 
87824 
4649 
5.29 


5862 
548766 
5386 
0.98 


1131 
135695 
5106 
3.76 


7812 
772285 
7030 
0.91 
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APPENDIX  A 
Instrument  and  Item  Tables 


Table  A.l 

Subject  Area  Blocks  in  Booklets,  Grade  3/Age  9 


BOOKLET  BLOCKS 


*1) 

9R1 

9M1 

9S1 

*2) 

9S2 

9R2 

9M3 

*3) 

9M2 

9S3 

9R3 

**4) 

9M1 

9M2 

9S3 

**5) 

9S1 

9S2 

9M3 

6) 

9S3 

9S4 

9S1 

7) 

9R5 

9R1 

9C2 

8) 

9S3 

9S7 

9S5 

9) 

9C2 

9M6 

9S3 

10) 

9R3 

9R2 

9C1 

11) 

9M4 

9M1 

9M2 

12) 

9R1 

9M4 

9S4 

13) 

9R4 

9C1 

9M3 

14) 

9S2 

9R6 

9M7 

15) 

9M1 

9C2 

9R6 

16) 

9M2 

9R6 

9S1 

17) 

9S6 

9M5 

9R1 

18) 

9S6 

9S3 

9S2 

19) 

9M1 

9M6 

9M3 

20) 

9S1 

9C2 

9M3 

21) 

9M5 

9S4 

9R3 

22) 

9M2 

9M5 

9M3 

23) 

9R3 

9S5 

9M3 

24) 

9R4 

9S3 

9M1 

25) 

9R4 

9S5 

9M6 

26) 

9R6 

9R3 

9R1 

BOOKLET 

BLOCKS 

27^ 

9S1 

9S5 

9S6 

28^ 

9M6 

9M2 

9M7 

29^ 

9M4 

9M6 

QMS 

30) 

9S4 

9C3 

31) 

9S7 

9C3 

9M5 

32) 

9P.5 

9M2 

9S7 

33) 

9C2 

9C1 

9C3 

34) 

9S5 

9M7 

9C1 

35) 

9R2 

9R5 

9R6 

36) 

9M5 

9M7 

9M1 

37) 

9C1 

9R1 

9S3 

38) 

9M6 

9S6 

9R2 

39) 

9S2 

9R3 

9M4 

40) 

9S5 

9S2 

9S4 

41) 

9S7 

9S1 

9S2 

42) 

9S4 

9S6 

9S7 

43) 

9R2 

9S7 

9C2 

44) 

9C3 

9S2 

9R5 

45) 

9M7 

9C3 

9R2 

46) 

9M7 

9M4 

9M3 

47) 

9M1 

9S1 

9R5 

48) 

9R1 

9R2 

9R4 

49) 

9R6 

9R4 

9C3 

50) 

9C1 

9M4 

9S6 

51) 

9R5 

9R4 

9R3 

Booklet  used  for  Bridge  A  assessment  only 
'  Booklet  used  for  Bridge  B  assessment  only 


-421- 


Table  A.  2 


Subject  Area  Blocks  in  Booklets,  Grade  7/Age  13 


BOOKLET  BLOCKS  BOOKLET  BLOCKS 


*1^ 

13R1 

13M1 

13S1 

35) 

13R4 

13S7 

13M3 

*- ) 

13S2 

13R2 

13M3 

36) 

13C6 

13R3 

13R2 

) 

13M2 

13S3 

13R3 

37) 

13S3 

13M2 

13S8 

13M1 

13M2 

13S3 

38) 

13M5 

13M6 

13M9 

13S1 

13S2 

13M3 

39) 

13S5 

13M4 

13R3 

13R3 

13R1 

13R5 

40) 

13C5 

13S1 

13M4 

13M7 

13M9 

13M3 

41) 

13S7 

13S6 

13S8 

1388 

13M5 

13R1 

42) 

13R2 

13M2 

13S4 

13C1 

13C5 

13C4 

43) 

13M2 

13M8 

13M9 

10^ 

13M4 

13M2 

13M5 

44) 

13M4 

13M1 

13M9 

11) 

13S8 

13S2 

13S4 

45) 

13C2 

13C1 

13R6 

12) 

13S8 

13C2 

13R4 

46) 

13S5 

13M6 

13C4 

13) 

13M2 

13M1 

13M3 

47) 

13R5 

13S9 

13M6 

14) 

13R6 

13R5 

13C5 

48) 

13C4 

13S6 

13M9 

15) 

13S3 

13S5 

13S7 

49) 

13S6 

13S9 

13S1 

16) 

13R3 

13C2 

13S6 

50) 

13C4 

13R6 

13M2 

17) 

13M7 

13R3 

13C5 

51) 

13C1 

13R2 

13M9 

18) 

13C5 

13C3 

13R2 

52) 

13S6 

13S3 

13S2 

19) 

13R4 

13C4 

13C6 

53) 

13C2 

13S3 

13M7 

20) 

13R6 

13R4 

13R3 

54) 

13S7 

13S2 

13S9 

21) 

13S9 

13S4 

13S3 

55) 

13S4 

13C6 

13M5 

22) 

13C3 

13M8 

13S9 

56) 

13M6 

13M7 

13M2 

23) 

13C6 

13C5 

13C2 

57) 

13M1 

13R6 

13S1 

24) 

13R4 

13C1 

13R1 

58) 

13S9 

13S8 

13S5 

25) 

13M5 

13M8 

13M3 

59) 

13M1 

13M5 

13M7 

26) 

13M1 

13R1 

13C3 

60) 

13R5 

13S7 

13C1 

27) 

13C3 

13C4 

13C2 

61) 

13S1 

13S8 

13S3 

28) 

13M7 

13M4 

13M8 

62) 

13S2 

13S1 

13S5 

29) 

13M6 

13M8 

13M1 

63) 

13S2 

13C3 

13M3 

30) 

13M8 

13R5 

13S2 

64) 

13C1 

13C6 

13C3 

31) 

13S4 

13S5 

13S6 

65) 

13S7 

13M7 

13M9 

32) 

13R1 

13R2 

13R6 

66) 

13S1 

13S4 

13S7 

33) 

13M4 

13M6 

13M3 

67) 

13R1 

13S3 

13C6 

34) 

13R2 

13R4 

13R5 

*  Booklet  used  for  Bridge  A  assessment  only 
**  Booklet  used  for  Bridge  B  assessment  only 
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Table  A. 3 

Subject  Area  Blocks  in  Booklets,  Grade  11/Age  17 
BOOKLET  BLOCKS  BOOKLET  BLOCKS 


1) 

Not  Used 

2) 

Not  Used 

3) 

Not  Used 

*4) 

17M1 

17M2 

17S3 

*5) 

17S1 

17S2 

17M3 

6) 

17S4 

17M11 

17M8 

7) 

13R6 

13R4 

1  7C1 

8) 

17S3 

13S5 

1.  t  O  £. 

9) 

13R^ 

13M5 

1  7M7 

10) 

17M8 

17M1 

^  ic.L 

J.  /  v^*+ 

11) 

13M5 

17M10 

J.  / 1  L\J 

12) 

17S9 

17M2 

13S6 

13) 

17M11 

13R3 

1  7M1  0 

J.  /  1 1  J.  v 

14) 

13R2 

13M4 

1  7M? 

15) 

17M2 

17S7 

13M4 

16) 

13S6 

17S4 

17M6 

17) 

17S8 

13S5 

17C6 

18) 

17S10 

17S9 

17S2 

19) 

17S1 

17S10 

17C2 

20) 

17M2 

17M6 

17M10 

21) 

17C6 

13R2 

17C1 

22) 

17C5 

17C3 

13R6 

23) 

17S9 

13S6 

17S8 

24) 

17M6 

13R5 

17C3 

25) 

17M6 

13M4 

17M9 

26) 

13R5 

13R2 

17C4 

27) 

17C2 

17C6 

17C3 

28) 

13R2 

17C5 

17S4 

29) 

17M7 

17M10 

17M9 

30) 

17M7 

17M2 

17C1 

31) 

17M1 

17M2 

17M3 

32) 

17M1 

17M7 

17M11 

33) 

17C5 

17C2 

13R4 

34) 

17M8 

13R1 

13S5 

35) 

17C2 

17C4 

13R6 

36) 

17M1 

13R2 

17M9 

37) 

17S11 

13M4 

17M9 

38) 

17C1 

17C3 

13R1 

39) 

17S11 

13S5 

13S6 

40) 

13M5 

17M6 

17M1 

41) 

17K6 

17M11 

17M3 

42) 

13R3 

17C2 

17M7 

43) 

13M5 

17M9 

17M3 

44) 

17M1 

17S7 

17S3 

45) 

17S3 

17C4 

13M5 

46) 

17C5 

17M11 

17M9 

47) 

17S1 

17S11 

13R1 

48) 

17S7 

17S3 

17S10 

'klet  used  for  Bridge  B  ass 


1  7C9 
1  /  oZ 

1  7  C  1  1 

1  /o  1  1 

SO") 

J.  J£\.  J. 

1  /C5 

51 

1  7<?7 

J.  /  o  / 

1  /  oo 

1  jRo 

J.  /  o  J 

1  7CQ 
1  /  o^ 

1  "7  C  "7 

j:>  ) 

J.  jrl  J 

1  '^KfA 

1  /Cz 

) 

1  7rA 

13R1 

S  S^ 
DD) 

1  /  o  / 

1  "7  C  1  1 

i  /oil 

17S4 

DO  ) 

1  '^C  R 
1  JO  J 

i  /SiU 

17S4 

0/ ) 

1  7Kn 

1  "7  C  O 

i  /Sz 

17M10 

1  /ra/ 

1  "7  C  0 

i  /So 

17S9 

Dy  ) 

1  /rlZ 

JL  /no 

17M9 

1  /rlO 

1  "7  C  1  1 
1  /Sii 

17M3 

1  '3D  R 

13R4 

1  7Q9 

17  c/. 

13R5 

1  /  OH 

1  7  C  Q 
i  /  OO 

1  "7  f  O 

1  /S3 

1  7M1  n 

1  /nl 

J.  Jo  O 

1  7*^1 

1/M9 

1  ICA 
L  /  KjH^ 

1  /C5 

67^ 

1  7M1  n 
i  /niU 

1  /M3 

1  7c: 

1  "711/0 

1  /ni 

J.  /  O  J.  J. 

1  7  c  1  n 
1  /  o  lU 

1  "7  C  Q 

1  /oo 

70^ 

J.  /  O  J 

1  /  o  11 

1  7  C  O 

1  /by 

71) 

J.  /  o  ^ 

1  7c:a 

1  /  0^+ 

1  7C1 
1  /  o  1 

72) 

17M8 

1  7M7 
1  /  m/ 

1  /no 

73) 

13M4 

1  7M7 

1  /rij 

74) 

13S5 

1  JriJ 

1  /  o  lU 

75) 

17C6 

1  '^p  ^ 

76) 

17M11 

1  7M9 

1  jXTIj 

77) 

17C1 

1  ICO 

1  Jf\.J 

78) 

17M11 

J.  /  O  J. 

1  7C1  n 

1  /  o  lU 

79) 

1  '^R4 

1  7rA 

1  7C1 
1  /  o  1 

80) 

J.  /  o  z. 

Uo  O 

1  7  C7 
1  /  o  / 

81) 

J.  J£\.  J 

1  '^PA 
1  jKM- 

1  OD  0 

1  jRz 

J/.  J 

J.  J  oO 

1  /  O  J 

1  7  C  1 

1  /o  1 

1  7M1  n 

J.  /  riJ.U 

1  7^^ 
1  /  Uo 

1  /JMj 

84^ 

J.  /  o  J. 

1  7C7 
1  /  o  / 

1  0  c  t^ 
1  Jo  J 

o 

1  '^R'^ 

1  QD^ 
1  jKO 

13R3 

86) 

J.  Jo  J 

1  7CQ 
1  /  o^ 

1  0  D  0 

liRi 

87^ 

Of) 

-I.  jn,'+ 

1  /  no 

1  "71/1  1 

1  /Mil 

88) 

I7S10 

13S6 

13R4 

89) 

13R6 

13R1 

13R2 

90) 

17C4 

17C3 

13R3 

91) 

17S2 

17S1 

17S8 

92) 

13R4 

17H1 

17L1 

93) 

17H2 

13R4 

17L2 

94) 

17L3 

13R4 

17H3 

95) 

17L4 

17H4 

13R4 

only 
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Table  A. 4 


Block- to -Block  Occurrence  Matrix,  Grade  3/Age  9 


Spiral  Booklets 


R1R2R3R4R5R6M1M2M3M4M5M6M7S1S2S3S4S5S6S7C1C2C3 

Rl  611111  11  11      1  11 

R2  61111                    11  11111 

R3  6111          111  111  1 

R4  61111          1  11  11 

R5  6111  11  111 

R6  611                 111  11 

Ml  611111111  1 

M2  6111111         1         1  1 

M3  611111  1  11 

M4  61111111 

M5  611  1111 

M6  6  11111 

M7  6  1111 

51  61111111 

52  6  111111 

53  6  111111 

54  6  1111 

55  6  111 

56  6  11 

57  6  11 
CI  6  11 
C2  6  1 
C3  6 


ERIC 
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Table  A. 5 


Block- to-Block  Occurrence  Matrix,  Grade  7/Age  13 


Spiral  Booklets 


R1R2R3R4R5R6M1M2M3M4M5M6M7M8M9S1S2S3S4S5S6S7S8S9C1C2C3C4C5C6 

Rl  6111111  1  1  111  1 

R2  611111                      1  1  1111 

R3  6111            11  11  111 

R4  6111  111111 

R5  61                  111  111  1 

R6  611                         1  1111 

Ml  6111111111  1 

M2  71111111         11  1  1 

M3  61111111  1  1 

M4  6111111  1  1 

M5  6111111  1 

M6  6111  111 

M7  7121111 

M8  6  11  11 

M9  7  1111 

51  611111111  1 

52  61111.  1111 

53  71111211  1 

54  611111  1 

55  6  11111 

56  6  11111 

57  7  111 

58  7  11 

59  6  1 

CI  6  11111 

C2  6  1111 

C3  6  111 

C4  6  11 

C5  6  1 

C6  6 
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Table  A. 6 


Block-to-Block  Occurrence  Matrix,  Grade  11/Age  17 
Spiral  Booklets* 


R1R2R3R4R5R6M1M2M3M4M5M6M7M8H9M;M: S1S2S3S4S5S6S7S8S9S ; S : C1C2C3C4C5C6 

Rl  711111  1             1  1  111111 

R2  71111111                 1  1  1  111 

R3  7111                        1          11  I  1  1111 

R4  711                 11                 1  1  11111 

R5  71                    1  11  111111 

R6  71  1  11  11111 

Ml  8111111112111  1  1 

M2  812  1111111  111  1 

M3  811111121  1  1  1  1 

M4  81111211  1  11 

M5  8111111  11  1  11 

M6  711111  11  1 

M7  81111  1111 

M8  811211  11 

M9  811  1  111 

M;  8  111  1 

M:  8  1111 

51  811111111211  1 

52  71111111111 

53  811121111  1 

54  •  81111111  1 

55  8111121  1 

56  8112111 

37  8  11111 

SB  8  2  111 

S9  8  111 

S;  8  11 

S:  8  1 

CI  7  11111 

C2  7  1111 

C3  7  111 

C4  7  11 

C5  7  1 

C6  7 

*Seraicolon  (;)  represents  10;  colon  (: )  represents  11, 
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Table >A.  7 

Composition  of  Items  in  Blocks,  Grade  3/Age  9 

#  #  Open- Ended 

Background        Cognitive  Total      Cognitive  Cognitive 

Mock      Type   Items   Items  Items  Items  Items 


9R1 

Rdg 

1-11 

12-21 

21 

10 

9R2 

Rdg 

1-9 

10-20 

20 

11 

9R3 

Rdg 

1-7 

8-17 

1  7 

1  n 

1 
1 

9R4 

RdR 

1-4 

5-16 

1  fi 

X  o 

x^ 

0 

9R5 

Rdg 

1-10 

11-24 

24 

14 

9R6 

Rdg 

1-4 

5-17 

17 

9S1 

Sci 

1-5 

6-23 

23 

9S2 

Sci 

1-25 

25 

9S3 

Sci 

1-11 

12-31 

31 

20 

9S4 

Sci 

1-9 

10-23 

23 

14 

9S5 

Sci 

1-4 

5-19 

19 

15 

1 

X 

9S6 

Sci 

1-4 

5-19 

19 

15 

1 

X 

9S7 

Sci 

1-7 

8-21 

21 

14 

9M1 

Math 

1-26 

26 

26 

9 

9M2 

Math 

1-26 

26 

26 

9 

9M3 

Math 

1-3 

4-19 

19 

16 

10 

(5-14  calc.) 

9M4 

Math 

1-7 

8-28 

28 

21 

7 

9M5 

Math 

1-11 

12-28 

28 

17 

9M6 

Math 

1-8 

9-28 

28 

20 

2 

9M7 

Math 

1-10 

11-28 

28 

18 

9C1 

Comp 

1-18, 

30, 

31 

19-29,32- 

-40 

40 

20 

1 

9C2 

Comp 

1-13, 

24, 

25 

14-23,26- 

■34 

34 

19 

2 

9C3 

Comp 

1-11,22,23 

12-21,24- 

■33 

33 

20 

1 
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Table  A. 8 

Composition  of  Items  in  Blocks,  Grade  7/Age  13 


# 

# 

Open- Ended 

Background 

Cognitive 

Total 

Cognitive 

Cognitive 

Block 

Type 

I  terns 

Items 

Items 

X  tems 

X  tems 

13R1 

Rdg 

1-19 

20-31 

31 

12 

1 

13R2 

Rdg 

1-9 

10-19 

19 

10 

13R3 

Rdg 

1-15 

16-18 

28 

13 

(19-28  SS)* 

13R4 

Rdg 

1-7 

8-21 

21 

14 

13R5 

Rdg 

1-6 

7-18 

18 

12 

13R6 

Rdg 

1-4 

5-18 

18 

14 

5 

13S1 

Sci 

i-  ii 

IZ  -  DO 

30 

13S2 

Sci 

1-9 

10-40 

40 

31 

13S3 

Sci 

1-9 

10-36 

36 

27 

13S4 

Sci 

1-9 

10-27 

27 

18 

1 

13S5 

Sci 

1 

2-19 

19 

18 

13S6 

Sci 

1-10 

11-28 

28 

18 

13S7 

Sci 

1-9 

10-27 

27 

18 

1 

13S8 

Sci 

1-14 

15-32 

32 

18 

1 

13S9 

Sci 

1-13 

14-31 

31 

18 

13M1 

Math 

1-14 

15-51 

51 

37 

9 

13M2 

Math 

1-7 

8-44 

44 

37 

8 

1  -ft 

8-^2 

32 

24 

10 

(9-24  calc.) 

13M4 

Math 

1-14 

15-43 

43 

29 

12 

13M5 

Math 

1-17 

18-43 

43 

26 

15 

13M6 

Math 

1-12 

13-48 

48 

36 

4 

13M7 

Math 

1-16 

17-55 

55 

39 

8 

13M8 

Math 

1-Z5 

16-58 

58 

63 

6 

13M9 

Math 

1-16 

17-57 

57 

41 

14 

(17-31  calc.) 

13C1 

Comp 

1-19,31,32 

20-30,33-42 

42 

21 

1 

13C2 

Comp 

1-13,25,26 

14-24,27-35 

35 

20 

2 

13C3 

Comp 

16-19 

1-15,20-28 

28 

24 

13C4 

Comp 

1-5,25,26 

6-24,27-37 

37 

30 

13C5 

Comp 

1-6,10,11, 

7-9,12-17 

30 

20 

18,19 

13C6 

Comp 

1-10,21,22 

11-20,23-33 

33 

21 

1 
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Table  A.  9 


Composition  of  Items  in  Blocks,  Grade  11/Age  17 

# 

#  #  Open -Ended 

Background       Cognitive  Total      Cognitive  Cognitive 

^lQ£k  — Items   Items  Items         Items  Items 


13R1 

Rdg 

1-19 

20-31 

31 

12 

1 

L 

13R2 

Rdg 

1-9 

10-19 

19 

10 

13R3 

Rdg 

1-15 

16-18 

(19-28  SS)* 

13R4 

Rdg 

1-7 

8-21 

21 

14 

13R5 

Rdg 

1-6 

7-18 

18 

12 

13R6 

Rdg 

1-4 

5-18 

18 

14 

1/rii 

Math 

1-14 

15-49 

49 

35 

10 

17M2 

Math 

1-14 

15-49 

49 

35 

17M3 

Math 

1-11 

12-35 

35 

24 

14 

(12-30  calc.) 

13M4 

Math 

1-14 

15-43 

43 

29 

12 

13M5 

Math 

1-17 

18-43 

43 

26 

15 

17M6 

Math 

1-10 

11-46 

46 

36 

1 8 

17M7 

Math 

1-16 

17-53 

53 

37 

17M8 

Math 

1-15 

16-52 

52 

37 

17M9 

Math 

1-20 

21-61 

61 

41 

13 

(21-33  calc.) 

17M10 

Math 

1-10 

11-46 

46 

36 

1  7M1  1 

naun 

1  11 
i-  11 

12-48 

48 

37 

3 

17S1 

Sci 

1-11 

12-38 

38 

27 

17S2 

Sci 

1-9 

10-41 

41 

32 

17S3 

Sci 

1-9 

10-32 

32 

23 

17S4 

Sci 

1-11 

12-31 

31 

20 

1 

13S5 

Sci 

1 

2-19 

19 

18 

13S6 

Sci 

1-10 

11-28 

28 

18 

17S7 

Sci 

1-17 

18-37 

37 

20 

1 

17S8 

Sci 

1-13 

14-33 

33 

20 

1 

17S9 

Sci 

1-17 

18-37 

37 

20 

1 

17S10 

Sci 

1-15 

16-35 

35 

20 

1 

17S11 

Sci 

1-9 

10-29 

29 

20 

1 

17C1 

Comp 

1-19,31,32 

20-30,33-44 

44 

23 

2 

17C2 

Comp 

1-13,25,26 

14-24,27-36 

36 

21 

3 

17C3 

Comp 

16-19 

1-15,20-28 

28 

24 

17C4 

Comp 

1-21,31,32 

22-30,33-40 

40 

17 

17CJ 

Comp 

1-18,33,34 

19-32,35-44 

44 

24 

17C6 

Comp 

1-17,26,27 

18-25,28-36 

36 

17 

1 
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Table  A. 9 
(continued) 


Background       Co  gn  i  t  ive 


Block 

Type 

Items 

Items 

17H1 

Hist 

1-12.49 

-61 

13-48 

17H2 

Hist 

1-12.49 

-61 

13-48 

17H3 

Hist 

1-12.48 

-60 

13-47 

17H4 

Hist 

1-12.47 

-59 

13-46 

17L1 

Lit 

1-18.49 

-72 

19-48 

17L2 

Lit 

1-18.50 

-73 

19-49 

17L3 

Lit 

1-18,49 

-72 

19-48 

17L4 

Lit 

1-18.49 

-72 

19-48 

#  #  Open- Ended 

Total      Cognitive  Cognitive 
Items         Items  Items 


61  36 

61  36 

60  35 

59  34 

72  30 

73  31 
72  30 
72  30 
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Table  A. 10 


Block  Occurrence  in  Booklets,  Grade  3/Age  9 
Occurs  in  Booklets 


Block 

DID 

Spiral 

9R1 

1 

7 

1  7 

J  /  , 

A  Q 

9R2 

0 

*+ J  , 

A  ^ 
J  . 

A  Q 

9R3 

'X 

Z  J  , 

jy . 

c  1 

51, 

LK* 

9R4 

ZD  , 

48, 

49. 

c  1 
Di 

9R5 

7, 

32, 

35. 

44, 

47. 

51 

9R6 

15, 

16, 

9^ 

49 

9S1 

1 

JL 

C 

D  , 

O  , 

-lo , 

on 

0  "7 

41, 

47 

9S2 

c 

D  , 

1  A 

, 

41, 

44 

9S3 

/. 

M- , 

/: 

D  , 

Q 

O  , 

Q 

1  Q 

24, 

37 

9S4 

c 

D  , 

01 

, 

A  n 

42 

9S5 

Q 

O  , 

27, 

34, 

40 

9S6 

17, 

18, 

27, 

38, 

42, 

50 

9S7 

8, 

31, 

32, 

AO 

43 

9M1 

1 1 

JLJL  , 

1  c 

» 

1  Q 

9/. 

A  "7 

9M2 

3  4 

1  1 

Jul.  y 

99 
ZZ  , 

28, 

30. 

'^0 

9M3 

2.  5 

13, 

19, 

20, 

22, 

23. 

46 

9M^ 

11, 

12. 

29, 

39, 

46. 

50. 

LM* 

9M5 

17, 

21, 

22, 

29, 

31. 

36 

9M6 

9, 

19. 

25, 

28, 

29. 

38 

9M7 

14, 

28, 

34, 

36, 

45. 

46 

9C1 

10. 

13. 

33, 

34, 

37. 

50 

9C2 

7, 

9. 

15, 

20. 

33. 

43 

9C3 

30, 

31. 

33, 

44. 

45, 

49 
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Table  A.  11 

Block  Occurrence  in  Booklets,  Grade  7/Age  13 
Occurs  in  Booklets 


Block 

Bridge 

BIB 

Spiral 

13R1 

1 

6, 

8. 

24. 

26. 

32. 

67, 

LM* 

13R2 

2 

18, 

32. 

34. 

36. 

42, 

51 

13R3 

3 

6, 

16. 

17. 

20. 

36, 

39 

13R4 

12. 

19. 

20. 

24. 

34, 

35 

13R5 

6, 

14. 

30, 

34. 

47, 

60 

13R6 

14, 

20. 

32, 

45. 

50, 

57 

13S1 

1. 

5 

40, 

49 . 

57 , 

61. 

62, 

r  c 

13S2 

2. 

5 

11, 

30. 

52, 

54. 

62, 

63 

13'-.3 

3. 

4 

15. 

21. 

37, 

52. 

53, 

61, 

67 

13S4 

11. 

21. 

31. 

42. 

55, 

66 

13S5 

15. 

31. 

39. 

46. 

58. 

62 

13S6 

16. 

31. 

41. 

48, 

49, 

52 

13S7 

15. 

35. 

41. 

54. 

60, 

65, 

66 

13S8 

8. 

11. 

12. 

37. 

41, 

58, 

61 

13S9 

21. 

22. 

47. 

49. 

54, 

58 

13M1 

1. 

4 

13. 

lb  , 

^'i , 

44. 

57, 

Dy 

13M2 

3, 

4 

10. 

13, 

37. 

42. 

43, 

50, 

56 

13M3 

2. 

5 

7, 

13. 

25. 

33. 

35, 

63 

13M4 

10. 

28. 

33. 

39. 

40, 

44 

13M5 

8. 

10. 

25. 

38. 

55, 

59 

13M6 

29. 

33. 

38, 

46. 

47, 

56 

13M7 

7. 

17. 

28, 

53. 

56, 

59. 

65. 

13M8 

22, 

25, 

28, 

29. 

30. 

43 

13M9 

7. 

38. 

43. 

44. 

48, 

51. 

65 

13C1 

9. 

24, 

45. 

51, 

60, 

64 

13C2 

12. 

16. 

23, 

27, 

45, 

53 

13C3 

18. 

22. 

26, 

27, 

63, 

64 

13C4 

9. 

19. 

27, 

46. 

48, 

50 

13C5 

9. 

14. 

17. 

18. 

23, 

40 

13C6 

19, 

23. 

36. 

55. 

64, 

67 
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Table  A. 12 


Block  Occurrence  in  Booklets,  Grade  11/Age  17 


Block 

13R1 
13R2 
13R3 
13R4 

13R5 
13R6 

17S1 

17S2 

17S3 

17S4 

13S5 

13S6 

17S7 

17S8 

17S9 

17S10 

17S11 

17M1 

17M2 

17M3 

13M4 

13M5 

17M6 

17M7 

17M8 

17M9 

17M10 

17M11 

17C1 
17C2 
17C3 
17C4 
17C5 
17G6 


Bridge 


5 
5 
4 


4 
4 
5 


RTR 

irax 

34, 

38, 

47, 

50, 

54, 

61. 

89, 

LM* 

1^, 

21, 

26, 

28, 

36, 

81. 

89 

13, 

^2, 

50, 

81, 

85, 

86, 

90 

7, 

9, 

33, 

61, 

79, 

81. 

88, 

92, 

94, 

95 

24, 

26, 

61, 

62, 

75, 

77, 

85 

7, 

22, 

35, 

51, 

68, 

85, 

89 

19, 

47, 

71, 

78, 

79, 

82. 

84, 

91 

8, 

18, 

49, 

57, 

62, 

80, 

91 

8, 

44, 

45 , 

48 , 

63 

DO  , 

/U  , 

oZ 

6, 

16, 

28^ 

5?' 

56  i 

62, 

63. 

71 

8, 

17, 

34, 

3>, 

56, 

74, 

84, 

86 

12, 

16, 

23, 

39, 

65, 

80, 

82, 

88 

13, 

44, 

48, 

51, 

52, 

55, 

80, 

84 

17, 

23, 

51, 

58, 

63, 

67, 

69, 

91 

12, 

18, 

23, 

52, 

58, 

70, 

71, 

86 

18, 

19, 

48, 

56, 

69, 

74, 

78, 

88 

37, 

39, 

47, 

49, 

55, 

60, 

69, 

70 

10, 

31, 

32, 

36, 

40, 

44, 

57, 

64 

12 , 

14, 

15, 

20, 

30, 

31, 

59, 

76 

31, 

41, 

43, 

60, 

67, 

68 

73 

83 

1^, 

15, 

25, 

37, 

53, 

64, 

73, 

87 

9, 

11. 

40, 

43, 

45, 

53. 

74. 

76 

16, 

20, 

24, 

25, 

40. 

41, 

72 

9, 

29, 

30, 

32, 

42. 

58. 

72, 

73 

6, 

10, 

11> 

34. 

59. 

60. 

72, 

87, 

25, 

29, 

36, 

37, 

43. 

46, 

59, 

65 

11 , 

13 , 

20, 

29 , 

57, 

64, 

67, 

83 

6, 

13, 

32, 

41, 

46, 

76, 

78, 

87 

7, 

21, 

30, 

38, 

•55, 

66, 

77 

19, 

27, 

33, 

35, 

42, 

53, 

77 

22, 

24, 

27, 

38, 

49, 

52, 

90 

10, 

26, 

35, 

45, 

54, 

66, 

90 

22, 

28, 

33, 

46, 

50, 

66, 

75 

17, 

21, 

27, 

54, 

75, 

79, 

83 
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Table  A. 12 
(continued) 


Occurs  in  Booklets 
Block  Bridge   BIB  Spiral 


17H1 

92 

17H2 

93 

17H3 

94 

17H4 

95 

17L1 

92 

17L2 

93 

17L3 

94 

17L4 

95 

■4^ 
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TableiA.13 
READING  COGNITIVE  ITEMS 


COHORT  1 


FIELD 

BLOCK 

ITEM 

N001501 

Rl 

17 

N001502 

Rl 

18 

N001503 

Rl 

19 

N00150A 

Rl 

20 

N002001 

R2 

lA 

N002002 

R2 

15 

N002003 

R2 

16 

N002801 

R2 

17 

N002802 

R2 

18 

N003001 

N003003 



N003101 

R3 

12 

N003102 

R3 

13 

N00310A 

R3 

lA 

N003105 

R3 

lA 

N003201 

N003202 



N003203 

N00320A 

NOOAlOl 

R2 

19 

N00A601 

-- 

N00A602 

N00A603 

N005001 

N005002 

N005003 

N005701 

-- 

N005702 

N005703 

N006001 

N006002 

N006003 

-- 

N007101 

N007102 

N007103 

N00710A 

N007301 

N007302 

-- 

N007303 

N00730A 

-- 

N007305 

N007306 

N007A01 

N007A02 

-- 

N007A03 

N007A0A 

-- 

N007A05 

N008201 

-- 

N008202 

N008203 

-- 

N00820A 

N008205 

N008601 

R3 

15 

N008602 

R3 

16 

N008603 

R3 

17 

N008901 

Rl 

15 

N008902 

Rl 

16 

N009A01 

R2 

13 

N009801 

R2 

11 

NOIO.IOI 

R5 

11 

N010102 

R5 

12 

N010103 

R5 

13 

N010201 

R2 

20 

N010301 

R2 

10 

NOlOAOl 

Rl 

12 

N010A02 

Rl 

13 

N010A03 

Rl 

lA 

N010501 

R3 

8 

N010502 

R3 

9 

N0105O3 

R3 

10 

N01050A 

R3 

11 

N010601 

R6 

10 

N010602 

R6 

11 

COHORT  2 


COHORT  3 


BLOCK 

ITEM 

T  Tr'ii 

lTr.n 

Rl 

25 

Rl 

Z  J 

Rl 

26 

Rl 

Z  0 

Rl 

27 

Rl 

9  7 

Rl 

28 

Rl 

2  8 

Rl 

22 

Rl 

Rl 

23 

Rl 

0 

Rl 

2A 

Rl 

2A 

Rl 

20 

Rl 

20 

Rl 

21 

Rl 

91 

R2 

18 

R2 

1  A 
X  0 

R2 

19 

19 

Rl 

29 

29 

Rl 

30 

p  1 

•5  ri 
J  U 

Rl 

31 

Rl 

J  X 

Rl 

31 

D  1 

J  X 

R5 

12 

R  ^ 
t\j 

1  9 
X  Z 

R5 

13 

R5 

X  J 

R5 

lA 

R5 

1 A 

R5 

15 

R5 

1 5 

R3 

16 

R3 

16 

R3 

17 

R3 

1 7 

R3 

18 

R3 

18 

R2 

15 

R2 

15 

R2 

16 

R2 

16 

R2 

17 

R2 

17 

R3 

22 

R3 

22 

R3 

23 

R3 

23 

R3 

2A 

R3 

2A 

R3 

19 

R3 

19 

R3 

20 

R3 

9n 

R3 

21 

R3 

2 1 

R3 

25 

R3 

2  5 

R3 

26 

R3 

26 

R3 

27 

R3 

27 

R3 

28 

R3 

28 

RA 

13 

RA 

13 

RA 

lA 

RA 

lA 

RA 

15 

RA 

1 5 

RA 

16 

RA 

16 

RA 

17 

RA 

X  / 

RA 

18 

RA 

18 

RA 

8 

R^ 

8 

RA 

9 

RA 

9 

RA 

10 

RA 

10 

RA 

11 

RA 

11 

RA 

12 

RA 

12 

R2 

10 

R2 

10 

R2 

11 

R2 

11 

R2 

12 

R2 

12 

R2 

13 

R2 

13 

R2 

lA 

R2 

lA 

FIELD 

N010603 

N01060A 

N010605 

N013001 

N013002 

N013003 

N01300A 

N013301 

K013A01 

N013402 

N013A03 

N01A201 

N01A301 

N01A302 

N01A303 

N021101 

N021102 

N021103 

N021201 

N021202 

N02-.203 

N02120A 

N021301 

N021302 

N021303 

N02130A 

N021305 

N021306 

N021308 

N021309 

N021A01 

N021A02 

N021A03 

N021A0A 

N021501 

N021502 

N021503 

N02150A 

N021505 

N021601 

N021602 

N021603 

N02160A 

N021605 

N021701 

N021702 

N021703 

N021801 

N021802 

N021803 

NOl  ^A 

N02i 

N0218UO 

N021810 


COHORT  1 
BLOCK  ITEM 


R6 
R6 
R6 
R5 
R5 
R5 
R5 
R2 
R6 
R6 
R6 
Rl 
R5 
R5 
R5 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
RA 
R5 
R5 
R5 
R5 
R6 
R6 
R6 
R6 
R6 


12 
13 
lA 
21 
22 
23 
2A 
12 
15 
16 
17 
21 
18 
19 
20 
5 
6 
7 
13 
lA 
15 
16 
8 
9 
10 
11 
12 
8 
9 
9 
lA 
15 
16 
17 
5 
6 
7 
8 
9 


COHORT  2 
BLOCK  ITEM 


RA 
RA 
RA 


R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 


R5 
R5 
R5 
R5 
R5 
R5 
R5 
R5 
R6 
R6 
R6 
R6 
R6 
R6 
R6 


19 
20 
21 


5 
6 
7 
8 
9 
10 
11 
12 
13 
9 
10 
10 


7 
8 
9 
10 
11 
16 
17 
18 
lA 
15 
16 
17 
18 
lA 
18 


COHORT  3 
BLOCK  ITEM 


RA 
RA 
RA 


R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 
R6 


R5 
R5 
R5 
R5 
R5 
R5 
R5 
R5 
R6 
R6 
R6 
R6 
R6 
R6 
R6 


19 
20 
21 


5 
6 
7 
8 
9 
10 
11 
12 
13 
9 
10 
10 


7 
8 
9 
10 
11 
16 
17 
18 
lA 
15 
16 
17 
18 
lA 
18 
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Table  A.IA 
MATHEMATICS  COGNITIVE  ITEMS 


COHORT  1 

COHORT  2 

COHORT  3 

COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

BLOCK  ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

N200101 

__ 





M8 

31 

N212101 

_  ^ 

Ml 

45 

N200201 



Ml 

26 

N212201 



Ml 

37 

N200401 

M6  27 

M6 

35 

N212301 



__ 

__ 

Ml 

40 

N200f01 

M7 

26 

N212501 

-- 

-- 

-- 

Ml 

47 

N200701 

__ 



M8 

48 

N212601 

•  _ 

-- 

Ml 

43 

N200702 

__ 



M8 

49 

N212701 

__ 

_- 

Ml 

21 

N200901 

__ 

M7 

44 

M8 

34 

N212901 

__ 

M9 

32 

M8 

21 

N201001 

M8 

57 

M8 

42 

N212902 

-- 

-- 

M9 

33 

M8 

22 

N201101 

__ 

M6 

42 

M7 

25 

N212903 

-- 

-- 

M9 

34 

M8 

23 

N201201 

__ 

M7 

53 

N213001 

__ 

M8 

27 

N201301 

_  _ 

M7 

49 

N213101 

__ 

M7 

36 

N201A01 

__ 

M7 

39 

M7 

23 

N213201 

-- 

__ 

-- 

MS 

29 

N201A02 

__ 

M7 

40 

M7 

24 

N213401 

-- 

-- 

-- 

M9 

49 

N201701 

__ 

M9 

45 

M8 

30 

N213501 

-- 

-- 

-- 

M9 

50 

N201801 

__ 

M6 

44 

N213601 

-- 

-- 

M8 

41 

M9 

40 

N202S01 

__ 

M7 

38 

M7 

27 

N213701 

-- 

-- 

Ml 

28 

N202701 

__ 

Ml 

40 

N214101 

-- 

-- 

Ml 

19 

N202801 

M7  15 





N214301 

M7 

13 

N203001 

__ 

M9 

56 

M8 

50 

N214501 

Ml 

30 

N203201 

__ 

M7 

45 

M7 

18 

N214701 

M8 

22 

M9 

39 

N203601 

__ 

M7 

43 

M7 

29 

N214801 

-- 

Ml 

38 

N203701 

__ 

M6 

43 

N214901 

M7 

27 

M7 

41 

M9 

45 

N203801 

__ 

M8 

37 

M9 

46 

N215001 

M7 

19 

M9 

39 

M9 

37 

N204101 

__ 

M7 

18 

M8 

16 

N215101 

Ml 

29 

N20A201 

^_ 

Ml 

42 

N215301 

M6 

26 

M6 

32 

N20A401 





Ml 

21 

N215401 

M7 

14 

-- 

N20A501 





M9 

48 

N215601 

M8 

36 

KB 

26 

M204601 

H6  13 

M6 

17 

N215701 

__ 

__ 

M8 

24 

M8 

18 

N204701 

M6 

46 

__ 

N216101 

__ 

__ 

M7 

33 

N204801 

-_ 

M6 

40 

_  _ 



N216201 

M9 

43 

N20A901 

M6  12 

M6 

16 





N216301 

M9 

51 

M7 

30 

N205001 

M6 

39 

N216401 

M9 

4g 

M7 

28 

N205101 

M8 

31 

Ml 

14 

N216501 

M6 

17 

M6 

21 

N20S201 

M7 

22 

Ml 

12 

N216601 

M6 

22 

M6 

26 

N205301 

M6  10 

M6 

14 

N216g01 

M7 

55 

M7 

36 

N205i)01 

Ml 

11 

N217101 

M8 

51 

M7 

34 

N205801 

-- 

-- 

Ml 

31 

N217201 

M6 

16 

M6 

20 

-- 

N205901 

-- 

M7 

42 

Ml 

22 

N217701 

-- 

-- 

M7 

52 

M7 

AO 

N206301 

M9 

58 

N2178Ci 

-- 

-- 

M7 

54 

M7 

38 

N206501 

-- 

M8 

27 

M8 

25 

N218501 

-- 

-- 

M8 

47 

M7 

20 

N206601 

M7  16 

M7 

28 

M8 

20 

N218801 

Ml 

22 

N206701 

-- 

M8 

46 

Ml 

26 

N219001 

-- 

Ml 

30 

N206801 

M8 

19 

M9 

38 

N219101 

M6 

11 

M6 

15 

-- 

'N207101 

M9 

36 

M8 

24 

N2193C1 

-- 

-- 

M8 

49 

M7 

19 

N207401 

M6  18 

M6 

22 

-- 

N219401 

-- 

Ml 

38 

N207501 

M6 

48 

-- 

-- 

N219501 

-- 

-- 

M7 

51 

N207601 

-- 

M6 

33 

N219701 

M9 

53 

Ml 

35 

N207701 

__ 

M7 

48 

N219901 

M5 

2A 

-- 

-- 

N207801 

M7  17 

M7 

24 

Ml 

15 

N220001 

M6 

23 

M6 

27 

N208X01 

__ 

M9 

41 

M8 

17 

N220101 

M6 

15 

rf6 

19 

N208301 

__ 

M8 

48 

Ml 

19 

N220201 

M7 

50 

N208401 

M6 

34 

N220301 

-- 

M9 

55 

N208501 

__ 

M8 

50 

M8 

47 

N220401 

-- 

-- 

M7 

A  9 

M7 

32 

N208601 

__ 

M9 

47 

M8 

38 

N220501 

M9 

40 

M7 

22 

N208801 

__ 

Ml 

16 

N220601 

-- 

-- 

-- 

M7 

52 

N208g01 

__ 

Ml 

34 

N220701 

-- 

-- 

-- 

M7 

37 

N20gi01 

M7  25 

M9 

48 

N220801 

-- 

-- 

-- 

Ml 

29 

N20g301 

__ 

Ml 

23 

N220901 

-~ 

-- 

M7 

53 

N209401 



_  ^ 

M8 

32 

N221001 

Ml 

36 

N209501 

M9 

43 

Ml 

18 

N221101 

-- 

-- 

-- 

-- 

Ml 

48 

N209601 

-- 

Ml 

17 

N221201 

M9 

56 

N209801 

M7  20 

M9 

46 

M9 

44 

N221601 

M9 

57 

N209901 

Ml 

24 

N221701 

M9 

54 

N210101 

M8 

46 

N221801 

M9 

61 

N210301 

Ml 

25 

N221901 

M9 

50 

M9 

47 

N210401 

Ml 

20 

N222101 

M8 

51 

N210601 

Ml 

14 

N222401 

M8 

21 

M9 

36 

N210701 

Ml 

16 

N222501 

M8 

39 

N210901 

M8 

36 

N222801 

M8 

33 

N211001 

Ml 

46 

N223001 

Ml 

33 

N211501 

Ml 

41 

M223101 

Ml 

46 

N211801 

Ml 

32 

N223301 

M8 

39 

M8 

45 

N2iigoi 

M9 

51 

K223601 

Ml 

43 

-436- 


ERIC 


(continued) 


COHORT  1 


COHORT  2 


COHORT  3 


r  ituitu 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

MZZ Jo  0 1 

Ml 

27 

I1ZZ4  J  0 1 

M5 

1 6 

— — 

0 1 

**** 

M6 

4 1 

— 

N224/0 1 

M4 

25 

~  - 

Nzz«»/Oz 

M4 

26 

-  — 

MB 

1 8 

M7 

17 

I1ZZ3UU  1 

M7 

44 

nzzo 1 

M8 

44 

nzzD*i  wi 

Ml 

39 

M8 

56 

M7 

39 

MOO       n 1 

M8 

23 

Ml 

17 

MoORn  n 1 

Ml 

18 

MOOfiO  n 1 

M7 

•1  / 

Ml 

25 

N226A01 

MO 

35 

MO  07 1 n 1 
^ / X  u  X 

Ml 

35 

H  A«  A«  /  ^  U  X 

no 

•1  / 

M007^ni 
n  ^  ^  /  <j  u  X 

M7 

31 

MO 07 A  n 1 

MS 

52 

MO  0  7  7  n 1 
nzz / / u  X 

M6 

20 

M6 

24 

~"* 

MO  0  T  0  n 1 

NZZ / 9  0  X 

M6 

31 

MOOftn  n 1 

ux 

0  T 
Z  / 

MO  0     n 1 
nzzo 1 

M7 

42 

NzzodOX 

M7 

37 

M8 

28 

MO  0  Q  T  n 1 

nzzo / u  X 

"7 

M9 

52 

MO  0  ft  0  n 1 
nzzo  ou  X 

M4 

28 

"~ 

MO  0  on  n 1 

M8 

29 

H9 

41 

MO '>Q^  n  1 
nz Z9  X  u  X 

M5 

21 

~" 

MO  0  QO  n 1 

nz  ZsZ  u  X 

M8 

42 

M7 

46 

MO  OQO  no 
nz  zszuz 

M8 

43 

M7 

47 

MO  OQO  n'^ 
n  ^  ^  9  ^  u  0 

M8 

4  4 

M7 

48 

MO  OQ'a  n  1 

M8 

45 

M7 

35 

N22960 1 

Ml 

4  4 

N229801 

M  1 

nl 

36 

N229901 

Ml 

nx 

'a  7 

N230001 

M  1 

n  1 

4  4 

n  £t    \j  ji  \j  ±, 

no 

1  X 
x% 

M6 

18 

MO'ino  ni 
n  £t    \j  ^  yj  ±, 

no 

35 

M8 

35 

MO  'a  fra  n  1 
n  z  0  u  0  u  X 

MS 

60 

M0 1 n  A  m 
nzo  U4  u  X 

Ml 

32 

MO  'a  n  ^  n  1 
n  z  0  u  D  u  X 

M6 

9 

M6 

13 

n  ^ u  D  u  X 

M9 

52 

M7 

43 

MO      7  n 1 

Ml 

28 

MO  Q  n  ft  n 1 
n  z  0  u  0  u  X 

~~ 

Ml 

4 1 

MO   1  1  n  1 
n  z  0  X  X  u  X 

M7 

27 

MS 

19 

MO'i  1  '»m 
n  ^ X u  X 

M9 

54 

M7 

45 

11  &i  w  X  ^  U  X 

Ml 

39 

11  &i  w  X      W  X 

M  ft 

no 

30 

Ml 

13 

&1  &i  w  X  /  w  ^ 

M6 

ZD 

no 

OQ 
Z9 

N23180i 

no 

'a  ft 
00 

f" 
Mo 

4  3 

N232001 

M  1 

nl 

27 

N232101 

M8 

4 1 

N232601 

M  1 

nl 

4  5 

N232901 

M7 

Ml 

15 

N233101 

Ml 

42 

N233401 

Mi 

nl 

23 

n  ^  w  w  ^  ^  A» 

Ml 

24 

MO    L  1  n  1 
nz  0  *«  X  u  X 

M7 

0  1 
Z  X 

n9 

37 

M9 

53 

MO  'a  xo  n  1 
n  z  0  *i  z  u  X 

M8 

20 

M7 

21 

MO  1 A 1  n  1 
nz  o*(  0  u  X 

M9 

55 

M7 

41 

MO  'ax  ^  n  1 
nz  0  ^  o  u  X 

no 

28 

M6 

36 

~~ 

N234901 

M7 

22 

M8 

'ao 
oz 

Mo 

0" 
37 

N235101 

M6 

30 

N235201 

M5 

22 

N235301 

M6 

38 

N235501 

M5 

19 

N235601 

M6 

21 

M6 

25 

N236101 

MS 

28 

M8 

40 

N236201 

M6 

37 

N236401 

M6 

19 

M6 

23 

H236501 

M7 

11 

H236901 

M7 

24 

N237301 

M6 

28 

N237401 

M6 

45 

COHORT 

1 

COHORT  2 

COHORT 

3 

FIELD 

BLOCK  ITEM 

BLOCK 

ITEM 

BLOCK  ITEM 

N237501 

MC 

no 

24 

**  — 

**  *• 

N237601 

n/ 

18 

**  *• 

N237701 

M^ 

nj 

23 



**" 

N238001 

M^ 

no 

i  f 

**** 

N238101 

M7 

n  / 

0  A 

zo 

N238201 

M^ 

no 

0  c 

Zo 

N238401 

M^ 

nj 

ZD 

N238701 

M^ 

nj 

1  0 
1 0 

N238901 

M^ 

nj 

1  A 
X  *l 

M239101 

M7 

1  0 

xz 

N239201 

M5 

1  0 
X  z 

N239301 

M5 

28 

N239401 

M5 

13 

N239501 

M7 

23 

N239601 

M7 

28 

-- 

-- 

N239801 

M5 

17 

N239901 

M5 

20 

N240001 

M5 

0  7 
z  / 

N250201 

MA 

nH 

1  Q 

X  9 

M2 

19 

N250301 

M2 

9  n 
z  u 

N250501 

M5 

39 

f  ~ 
M5 

39 

N250601 

MO 

nz 

13 

N250602 

MO 

nz 

1 4 

— 



N250603 

Mic 

15 

__ 

N250701 

Ml 

y 

M2 

14 

N250702 

M1 

n  X 

ft 
0 

M2 

15 

N250703 

M 1 
n  X 

Q 

9 

M2 

16 

~~ 

N250801 

M4 

16 

M4 

16 

N250802 

M4 

17 

MA 

n*i 

X  / 

N250803 

M4 

18 

MA 

n*» 

1  0 
Xo 

N250901 

M2 

17 

Ml 

25 

M6 

1  1 

X  X 

N250902 

M2 

18 

Ml 

26 

M6 

12 

N250903 

M2 

19 

Ml 

27 

M6 

13 

N251101 

Ml 

AQ 

*i  y 

N251201 

M5 

26 

M^ 

no 

oc 

zo 

N251401 

M2 

16 

N251601 

M4 

13 

N251701 

M8 

40 

M2 

4 1 

N251801 

M5 

32 

M5 

32 

N251901 

M7 

26 

N252001 

M2 

25 

M2 

40 

N252101 

Ml 

25 

Ml 

41 

N252201 

M5 

30 

ou 

N25260: 

Ml 

26 

M5 

40 

ii  n 
•1  yj 

N252701 

M8 

55 

MR 
no 

N252901 

M4 

23 

Ml 

32 

N253201 

M4 

42 

M4 

AO 
•IZ 

N253202 

_  ^ 



M5 

37 

M5 

37 

N253701 

M2 

22 

NzdSoO 1 



M5 

42 

M5 

42 

kl  0  C  ^  A  A  1 

N25390 1 



-- 

Ml 

39 

kl  0  C  ^  A  A  ^ 

N253902 

-- 

Ml 

40 

kl      f  ^  A  A  ^ 

N253903 





Ml 

41 

N253904 



Ml 

42 

N254001 

M3 

28 

MO 

nz 

01 
zx 

N254301 

M7 

35 

Ml 

nx 

00 

N254501 

M5 

35 

M^ 

no 

OJ 

N254601 

Ml 

16 

MO 

nz 

N254602 



Ml 

46 

Ml 

27 

N255101 

M4 

38 

M4 

38 

N255301 

M2 

46 

kl  0  C  f  ^  A  ^ 

N255302 

M4 

37 

M4 

37 

N255401 

M6 

43 

N255501 

M3 

33 

N255601 

M2 

45 

N255701 

Ml 

50 

Ml 

32 

N255801 

M2 

49 

N255901 

Ml 

33 

N255902 

Ml 

34 

N256001 

M3 

34 

N256101 

M2 

17 

Ml 

15 

N256301 

M4 

ic. 

M4 

19 

N256501 

M3 

30 

M6 

35 

-*37- 
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Table  A.IA 
(continued) 


COHORT  1  COHORT  2  COHORT  3  COHORT  1  COHORT  2  COHORT  3 

FIELD        BLOCK     ITEM  BLOCK     ITEM  BLOCK     ITEM  FIELD        BLOCK     ITEM  BLOCK     ITEM  BLOCK  ITEM 


N9  ^Rftn 1 

MA 

21 

M3 

32 

Ml 

36 

N267301 

MA 

12 

_  _ 

0 1 

M3 

35 

N267601 

Ml 

3 



1 

act  D  t  Ctw  X 

Ml 

11 

N2.67602 

Ml 

18 

n  £t  ^  /  *i  w  X 

M7 

23 



N267801 

Ml 

20 

N25760 1 

Ml 

35 

N267901 

__ 

M5 

Al 

M5 

Al 

MO  ^770 1 

MA 

22 

__ 

__ 

N267921 

M3 

30 

MO  S7 AO  1 

M2 

3 

._ 



N268201 

Ml 

2A 

M8 

16 

MO  ^7Qn 1 

M5 

23 

M5 

23 

N268221 

M3 

lA 

M9 

20 

MO  Kaon  1 

MA 

39 

MA 

39 

N268801 

M2 

A8 

n^jOj\j  X 

M3 

19 

N268901 





M2 

A7 

MO  <;nan  1 

Ml 

38 

M2 

38 

N269001 

M2 

25 

Ml 

AA 

M2 

22 

MO  *;ftA07 

M2 

31 

Ml 

26 

N269101 

Ml 

23 

M2 

26 

MO  <;aan  T 

M2 

A 1 

Ml 

37 

N269201 

M2 

AA 

M6 

Al 

MO  *;ftft04 
n  ^  ^  o  o  V/  M 

M7 

20 

Ml 

18 

N269A01 





M9 

57 

M9 

59 

H  '5  r  A  Q  n  1 

MA 

3 1 

MA 

3 1 

N269901 



M3 

29 

MO  *iQOfl  1 
n  £t  ^  9  w  w  X 

M2 

31 

N270001 

Ml 

lA 

MO  *iQ1  n  1 

MA 

16 

N270301 

M2 

20 

Ml 

30 

MO  *;q  *;  n  i 

M5 

36 

M5 

36 

N270302 



M2 

21 

Ml 

31 

11  ^  ^  9  9  U  X 

M7 

3A 

Ml 

28 

N270701 

M6 

37 

MO  «;qqo  1 

n  £t  ^  9  9  ^  X 

M3 

20 

M3 

22 

N270702 

M6 

38 

N260101 

LI  ^      V  X  V  X 

Ml 

A3 

M2 

20 

N270901 

1 

N260301 

__ 

M8 

53 

N270902 

MA 

15 

N260601 



M7 

21 

Ml 

16 

N271101 

M2 

2A 

M7 

17 

N260701 



M5 

33 

H5 

33 

N271301 

M9 

AA 

M3 

32 

N260801 

__ 



M2 

A3 

N271A01 

MA 

33 

MA 

33 

N260901 

__ 

M8 

5A 

Ml 

35 

N272101 

M3 

17 

-- 

-- 

N260902 

__ 

M^ 

AO 

MA 

AO 

N272102 

Ml 

15 

-- 

N261001 

Ml 

A7 

M2 

AO 

N272301 

M2 

1 

_- 

-- 

Ll  ^  \^  X  ^  ^  X 

M2 

38 

M2 

26 

N272302 

MA 

11 

-- 

N261301 

M2 

37 

M2 

318 

N272601 

MA 

17 

-- 

N261A01 

M2 

12 

N272801 

M3 

15 

N261501 

M2 

3A 

M2 

2A 

N273501 

M2 

6 

11  ^      X      V  X 

__ 

M2 

36 

M2 

27 

N273901 

Ml 

37 

M6 

36 

MORI  on  1 

n  ^  V  X  o  X 

M2 

35 

M2 

25 

N273902 



M5 

25 

M5 

25 

MOfiOnO 1 

M9 

A2 

N27A101 



M7 

25 

MOfioon 1 

Ml 

10 

M2 

18 

N27A801 



Ml 

29 

M6 

25 

ill  &i     ^  w  V  X 

M7 

19 

M2 

17 

N27A802 

M5 

29 

M5 

29 

ill  ^     ^  *f  V  X 

M3 

18 

Ml 

28 

Ml 

17 

N275001 

_  _ 

Ml 

A2 

N262501 

Ml 

19 

Ml 

33 

M2 

35 

N275301 



M3 

25 

-- 

-- 

N262502 

Ml 

20 

Ml 

3A 

M2 

36 

N275A01 

M2 

7 

-- 

-- 

-- 

N26260 1 

M7 

A6 

Ml 

38 

N276001 

M2 

21 

-- 

-- 

N262701 





MA 

15 

MA 

15 

N276002 

M2 

22 

-- 

N26280 1 

^_ 

__ 

MA 

20 

MA 

20 

N276021 

M3 

9 

N262802 





M 

21 

MA 

21 

N276022 

M3 

10 

-- 

-- 

-- 

N262803 



M*. 

22 

MA 

22 

N276101 

Ml 

12 

N263C01 





M7 

51 

Ml 

A3 

N276501 

MA 

18 

N263 101 



Ml 

39 

M2 

37 

N276601 

M2 

2 

N263201 





M2 

18 

N276801 

Ml 

A 

Ml 

17 

M6 

17 

N263202 

__ 

_  ^ 

_  ^ 



M2 

19 

N276802 

Ml 

5 

Ml 

18 

M6 

18 

N263A01 

M2 

A 

M2 

12 

N276803 

Ml 

6 

Ml 

19 

M6 

19 

N263A02 

M2 

5 

M2 

13 

N276821 

M3 

A 

M3 

9 

M3 

12 

N263501 

MA 

2A 

M2 

30 

M6 

3A 

N276822 

M3 

5 

M3 

10 

M3 

13 

N263801 

MA 

A3 

MA 

A3 

N276823 

M3 

6 

M3 

11 

M3 

lA 

N263901 



MA 

30 

MA 

30 

N277A01 

Ml 

2 

M2 

8 

N26A301 

M8 

58 

Ml 

A7 

N277501 

M2 

8 

N26A321 



__ 

M9 

28 

M3 

29 

N277601 

M2 

9 

Ml 

20 

M6 

20 

N26A501 

__ 



M7 

29 

N277602 

M2 

10 

Ml 

21 

M6 

21 

N26A521 

M3 

19 

M3 

23 

N277603 

M2 

11 

Ml 

22 

M6 

22 

N26A601 

MA 

3A 

MA 

3A 

N277621 

M3 

11 

M9 

17 

M9 

2A 

N26A701 

M2 

33 

M2 

39 

N277622 

M3 

12 

M9 

18 

M9 

25 

N265201 

MA 

lA 

Ml 

36 

N277623 

M3 

13 

M9 

19 

M9 

26 

N265202 

Wl 

30 

N277901 

MA 

8 

M2 

9 

M6 

lA 

N265A01 

Ml 

21 

N277902 

MA 

9 

M2 

10 

M6 

15 

N265901 

Ml 

AO 

M6 

39 

N277903 

MA 

10 

M2 

11 

M6 

16 

N265902 

M3 

31 

M6 

A2 

N278301 

MA 

35 

MA 

35 

N265903 

M5 

31 

M5 

31 

N278302 

MA 

36 

MA 

36 

N266001 

M5 

38 

M5 

38 

N278501 

M7 

30 

Ml 

23 

N266101 

Ml 

22 

M3 

27 

M6 

2A 

N278502 

M7 

31 

Ml 

2A 

N266501 

M3 

31 

N278503 

M7 

32 

Ml 

25 

N266701 

MA 

32 

MA 

32 

N278901 

M2 

32 

M2 

23 

N266801 

Ml 

31 

M2 

16 

N278902 

M2 

29 

M2 

A2 

N267001 

M3 

16 

N278903 

M2 

A2 

M2 

AA 

H267201 

Ml 

23 

N27890A 

Ml 

A9 

M6 

A5 

-A38- 
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T«ble  A. 14 
(contlnu«d) 


COHORT  1 


FIELD 

BLOCK 

ITE 

N278905 

_  _ 

— — 

N278921 



N278922 

_  _ 

N278923 



N278924 



N278925 



N279301 



N279321 



N279A01 





N279421 

_ 

N280401 

- 

N280A21 



N280601 



N280602 



N280603 



N280604 

N28060S 

N280606 



N280621 



N280622 



N280623 



_  _ 

N28062A 



N280625 

N280626 

N281A01 



N281901 

N282201 

-- 

N2o2202 

N282701 

_  ^ 

II 

N282801 

-- 

nzoz^u 1 

MA 

20 

N283001 

MO        1  1 

rCZoo  lU  1 

N28A001 

Ml 

16 

Ml 

17 

N284021 

M3 

7 

N28A022 

M3 

8 

N28A101 

N284102 

N28A401 

N28AA21 

__ 

N28A501 

N28A502 

N28A503 

N28A521 

-_ 



N28A522 

N28A523 

N285001 

N285021 

N285201 

N285301 

N285321 

N285A01 

N285701 

N285901 

N286001 

N286002 

N286101 

HI 

13 

N286102 

M2 

23 

N286201 

N286301 

N286302 

N286501 

N286502 

N286601 

N286602 

N286603 

COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


M8 
M3 
M3 
M3 
M3 
M3 
M9 
M5 

M9 
M5 
M9 
Md 
M9 
MA 
MA 
MA 
MA 
MA 
MA 
M3 
M3 
M3 
M3 
M3 
M3 
M2 
Ml 
M2 
M3 
M5 


M9 
Ml 


MS 
M5 
M5 

M9 
M5 

M5 

M5 

M9 
M9 
M9 
M5 
M9 
MA 
M7 
M9 
MA 
M2 


M8 
Ml 
Ml 

Ml 

M2 
M2 
M2 
M2 


52 
18 
22 
21 
2A 
23 
30 
3A 

29 
A3 
31 
33 
26 
23 
2A 
25 
26 
27 
28 
12 
13 
lA 
15 
16 
17 
39 
15 
28 
26 
2A 


A2 
51 


18 
19 
27 

2A 
20 

11 

22 

21 
22 
23 
28 
25 
29 
50 
27 
Al 
27 


17 
2A 
A5 

A8 
A3 
23 
2A 
25 


Ml 
M3 
M3 
M3 
M3 
M3 

Ml 
M5 
M9 
M5 
M9 
.:2 
M9 
MA 
MA 
MA 
MA 
MA 
MA 
M3 
M3 
M3 
M3 
M3 
M3 
M2 

M6 
M9 
M5 
Ml 


M6 
Ml 
Ml 
M9 
M9 
M5 
M5 
M6 
M5 
M9 
M6 
M5 
M6 
M5 
M6 
M5 
M9 
M9 
M9 
M5 
M9 
MA 

M3 
MA 
Ml 
M6 
Ml 
Ml 


M6 
M2 
Ml 
M2 
Ml 
M6 
M6 
M6 


AA 
21 
2A 
26 
28 
27 

31 
3A 
29 
A3 
30 
30 
28 
23 
2A 
25 
26 
27 
28 
15 
16 
17 
18 
19 
20 
29 

27 
3A 
2A 
A8 


AO 
12 
13 
32 
33 
18 
19 
26 
27 
27 
31 
20 
32 
21 
33 
22 
21 
22 
23 
28 
31 
29 

25 
Al 
21 
A6 

iq 

20 


23 
33 
22 
3A 
3A 
28 
29 
30 


FIELD 

N287101 
N287102 
N287301 
N287302 


COHORT  1 
BLOCK  ITEM 


COHORT  2 
BLOCK  ITEM 

M8  3A 

M9  38 

M8  25 

M8  26 


COHORT  3 
BLOCK  ITEM 

Ml  2d 

M2  32 

Ml  A5 

Ml  46 


-A39- 


ERLC 


Q 


Table  A. 15 
SCIENCE  COGNITIVE  ITEMS 


FIELD 


COHORT  1 
BLOCK  ITEM 


N400001 

N400101 

NA00102 

N400201 

N400301 

NAOOAOl 

NA00A02 

N400A03 

N40040A 

KA00405 

NA00501 

NA00601 

N400701 

NA00901 

NAOlOOl 

NAOllOl 

NA01201 

NA01301 

N4C1501 

NA01601 

NA01701 

NA01702 

NA01703 

NA01601 

NA01602 

NA01603 

K401804 

NA01901 

NA02001 

NA02002 

NA02003 

NA0200A 

■■';02005 

NA02101 

N402201 

NA02A01 

NA02501 

NA02601 

NA02602 

NA02603 

NA02701 

NA02601 

N402901 

N403001 

N403101 

N403201 

N403202 

N403301 

N403401 

N403501 

N403502 

N403503 

N403601 

N403701 

N403702 

N403703 

N403601 

N403802 

N403803 

N403604 

N403901 

M404001 

N404201 

N404501 

N404601 

N404701 

N404702 

N404601 

N404602 

N404603 

N404901 

N404902 

N404903 


SI 

SI 

SI 

SI 

SI 

SI 

SI 

SI 

SI 

SI 

SI 

Si 

SI 

SI 

SI 

SI 

SI 

SI 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S2 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 

S3 


6 
15 
16 
7 
6 
9 
10 
11 
12 
13 
14 
17 
16 
19 
20 
21 
22 
23 
1 
2 
3 
4 
5 
6 
7 
6 
9 
10 
11 
12 
13 
14 
15 
}6 
17 
16 
19 
20 
21 
22 
23 
24 
25 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
26 
29 
30 
31 


COHORT  2 
BLOCK  ITEM 


SI  16 


COHORT  3 
BLOCK  ITEM 


SI  12 


SI  26 


SI  30 


SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 


12 
13 
14 
15 
20 
21 
22 
17 
16 
19 


SI 


13 


FIELD 

N405001 

N405101 

N405201 

N405301 

N405401 

N405501 

N405601 

N405701 

N405601 

N405901 

N406001 

N406101 

N406201 

N406301 

N406302 

N406303 

N406304 

N406401 

N406402 

N406403 

N406404 

N406405 

N406501 

N406601 

N406701 

N406601 

N406602 

N406603 

N406604 

N406605 

N406606 

N406901 

N407OO1 

N407101 

N407201 

N407301 

N407302 

N407401 

N407402 

N407403 

N407404 

N4075"! 

N407601 

N407701 

N407601 

N407901 

N408001 

N406101 

N408201 

N406301 

N406302 

N406303 

N406304 

N406401 

N406501 

N406502 

N406601 

N406701 

N406601 

N408901 

N406902 

N406903 

N406904 

N409001 

N409101 

N409102 

N409103 

N409201 

N409301 

N409401 

N409402 

N409403 

N409501 


COHORT  1 

COHORT  2 

COHORT  3 

BLOCK  ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

SI 

23 

SI 

29 

SI 

24 

S3 

14 

__ 

SI 

25 

SI 

31 

__ 

SI 

26 

SI 

27 

S3 

19 

SI 

29 

S3 

21 

SI 

30 

-- 

-- 

__ 

SI 

31 

SI 

32 

__ 

SI 

33 

SI 

34 

SI 

33 

SI 

35 

SI 

35 

-- 

SI 

36 

SI 

37 

__ 

S2 

10 

SI 

21 

_  - 

S2 

11 

SI 

22 

S2 

12 

SI 

23 

S2 

13 

SI 

24 

S2 

14 

S2 

10 

S2 

15 

S2 

11 

S2 

16 

S2 

12 

S2 

17 

S2 

13 

S2 

16 

S2 

14 

-- 

S2 

19 

S2 

20 

SI 

28 

S2 

21 

S2 

22 

32 

16 

S2 

23 

S2 

17 

S2 

24 

S2 

16 

S2 

25 

S2 

19 

-- 

S2 

26 

S2 

20 

__ 

S2 

27 

S2 

21 

__ 

S2 

26 

S2 

27 

S2 

29 

S2 

33 

S2 

30 

S2 

36 

— 

S2 

31 

S2 

32 

S2 

32 

S2 

36 

S2 

33 

S2 

37 

__ 

S2 

26 

-- 

-- 

S2 

29 

S2 

30 

__ 

S2 

31 

-- 

S2 

36 

S2 

35 

S2 

37 

S2 

35 

S2 

36 

- 

-- 

S2 

39 

S2 

34 

__ 

-- 

SI 

38 

S2 

40 

S3 

10 

S3 

10 

S3 

11 

S3 

11 

S3 

12 

S3 

12 

S3 

13 

S3 

13 

S3 

14 

S3 

15 

S3 

16 

S3 

17 

SI 

19 

S3 

18 

S3 

19 

S3 

24 

S3 

20 

S3 

15 

S3 

21 

S3 

16 

S3 

22 

S3 

17 

S3 

23 

S3 

18 

S3 

24 

S3 

25 

S3 

26 

S3 

27 

S3 

28 

S3 

29 

SI 

20 

S3 

30 

S3 

31 

S3 

32 

S3 

33 

SI 

34 

-440- 
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Table  A. 15 
(continued) 


COHORT  1  COHORT  2  COHORT  3 


FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

NA09601 

S3 

3A 

-  - 

NA0970 1 

—  — 

~  ** 

S3 

35 

— 

N409801 

S3 

36 

-  - 

— 

N409901 

*"  ~ 

~  •" 

SI 

18 

N4 10001 

SI 

lA 

NA 10002 

~~ 

SI 

15 

NA10003 

—  ~ 

~  - 

SI 

IS 

NA 10004 

~  ** 

SI 

17 

N4 10101 

~  *" 

~~ 

SI 

25 

U  A  1  A 1  AO 

N«i  lOlOZ 

**" 

SI 

26 

NA lulu J 

**" 

SI 

27 

MAI  ft  9  ft  1 

SI 

32 

W/i  1  ft  T  ft  1 

hh 1 u j  u 1 

SI 

36 

NL 1  ft  A  ft  1 

S2 

15 

Mi  1  ft  ^  ft  1 
n*t  lU  DU  1 

**** 

S2 

22 

Mi  1  ftRft  1 

**** 

S2 

23 

11  A  1  A  C  A  o 

"** 

S2 

2A 

Mi  1  ft<?ft  O 

S2 

25 

Mi  1  ftf^fti 

S2 

26 

Mi  1  ft?  ft  1 

n**  lu  /  u  1 

S2 

3A 

Mi  1 n  fl  ft 1 
n*t  luou  1 

S2 

39 

Mi  1  ft  on 1 

S2 

AO 

M  /.  1  1  ft  A  1 

Nh 1 lUU 1 

S2 

Al 

N4 11101 

**" 

S3 

22 

U  A  1  1  0  A  1 

MA  1 IZO 1 

~" 

S3 

23 

U  A  1  1  0  A  1 

MA  1 loOl 

S3 

20 

U  A  1  1  A  A  1 

MA  1 lAOl 

S3 

25 

NA 1 1501 

S3 

26 

Mi  1  1  ^ft  0 
M«l  1 1  DUZ 

S3 

27 

Mi  1  1 Rft 1 

S3 

28 

Mi  1  1 7  ft 1 

S3 

29 

Mi  1  1  Aft 1 

n  *4  X  X  0  u  1 

S3 

30 

NA  11901 

S3 

31 

Mi  1  Oftftl 
afx^yjyji. 

"7 

S3 

32 

NA  12101 

Qi 

10 

SA 

1 0 

Mi  1 OOftl 
n  ^  X  ^  ^  u  X 

^  A 
0^ 

X  X 

S  A 

11 

Mi  1  5  q  ft  1 
n*i  x^o  u  X 

Qi 

9 1 

Z  X 

S  A 

21 

NA  l?*i01 

ii  *t  X  ^  ^  V  X 

S6 

1  Q 

xo 

Mi  1  0  Rft 1 

n**  xzou  X 

SA 

2A 

Hi  1  0  7ft  1 

ns  xz / u  X 

SA 

22 

SA 

22 

Mi  1 0  a  ft 1 

SA 

lA 

SA 

lA 

Mi  1  0  Oft  1 

M*l  IZ9U 1 

SA 

13 

SA 

13 

** " 

Mi  1  Q  ft  ft  1 
M*l  1 J  UU  1 

S9 

16 

Hi  1     1  ft  1 

X J  X  u  X 

S9 

19 

UA  1  1  0  A  1 

MA  1 jZuI 

SA 

17 

SA 

17 

— 

— 

Hi  1  "3  "3  ft  1 

£1*1 1 J  J  U  1 

SA 

1 8 

SA 

18 

— 

Mi  1  ^  /.  ft  1 

n  H  X  c  « u  X 

SA 

19 

SA 

19 

— 

Hi  1  Q  C  ft  1 

n*i  X  JbU  1 

SA 

12 

SA 

12 

— 

Mi  1  Q  <?  ft  0 
aH X JOUZ 

SA 

12 

SA 

12 

Hi  1  T  7ft  1 
n*4  X  J  /  U  X 

SA 

23 

S9 

18 

S 1 

12 

Hi  1     Oft  1 

n*t  X  J  9U  X 

S5 

5 

~~ 

Hi  1 i ftftl 
at  X 

SA 

25 

NA  lA  0 1 

Q  ^ 
OD 

6 

Hi  1  A  9ft  1 

n  *f  X  ^  X 

Q  ^ 
OD 

19 

Ha  1  i  q  ft 1 
JklH  X  H  0  U  X 

Q  ^ 
OD 

13 

HA lA AOl 

Q*i 

X  / 

S6 

23 

S6 

23 

NA 1 A  501 

Q  ^ 
OD 

1  R 
xo 

Hi  1  i  Rft  1 
n  *«  X  *4  V  u  X 

Q  ^ 
OD 

X  D 

NA  lA  701 

0  D 

1  0 

xz 

S7 

12 

Hi  1  i  flftl 
n  *t  X  *4  0 1/  X 

Q^ 

a 
0 

NA  lA9ni 

~  X  ^  9  V  X 

S5 

7 

NA15101 

S5 

lA 

NA15A01 

S6 

11 

NA15501 

SA 

16 

SA 

16 

NA15601 

S6 

18 

NA15701 

S6 

lA 

NA15801 

S6 

6 

NA16001 

S6 

7 

NA16101 

S6 

16 

NA16301 

S6 

10 

NA16A01 

SA 

26 

NA16501 

S6 

12 

NA16601 

S6 

19 

COHORT  1  COHORT  2  COHORT  3 

FIELD  BLOCK     ITEM  BLOCK     ITEM  BLOCK  ITEM 


NA16701 

S6 

15 

NA16702 

S6 

15 

NA16801 

S6 

25 

S6 

25 

NA16901 

S6 

2A 

S6 

2A 

NA17001 

-- 

S6 

18 

S6 

18 

NA17101 

S6 

27 

S6 

27 

NA17201 

S6 

28 

S6 

28 

NA17301 

S6 

lA 

S6 

lA 

NA17A01 

S9 

30 

SI 

26 

NA17601 

S6 

13 

S6 

13 

NA17701 

-- 

S6 

21 

S6 

21 

NA17801 

S6 

19 

S6 

19 

NA17901 

-- 

S6 

26 

S6 

26 

NA18001 

— 

S6 

15 

S6 

15 

NA18101 

S6 

22 

S6 

22 

NA18201 

S6 

17 

S6 

17 

NA18301 

S6 

12 

S6 

12 

NA18A01 

-- 

S6 

11 

S6 

11 

NA18501 

-- 

S6 

20 

S6 

20 

NA18701 

-- 

S5 

11 

S5 

11 

NA18702 

-- 

S5 

12 

S5 

12 

NA18801 

-- 

S5 

18 

S5 

18 

NA18901 

S5 

5 

S5 

5 

NA19001 

-- 

S5 

8 

S5 

8 

NA19101 

S5 

13 

S5 

13 

NA19201 

S5 

A 

S5 

A 

NA19301 

S5 

7 

S5 

7 

NA19A01 

— 

S5 

9 

S5 

9 

NA19501 

— 

S5 

3 

S5 

3 

NA19601 

— 

S5 

19 

S5 

19 

NA19701 

S5 

16 

S5 

16 

NA 19801 

— 

S5 

2 

S5 

2 

NA 19901 

— 

— 

S5 

17 

S5 

17 

NA20001 

— 

— 

S5 

15 

S5 

15 

NA20101 

— 

S5 

10 

S5 

10 

NA20201 

S5 

lA 

S5 

lA 

NA20301 

— 

— 

S7 

23 

NA20A01 

— 

S7 

19 

-- 

NA20501 

— 

S7 

20 

-- 

NA20601 

— 

— 

S7 

13 

NA2070X 

S7 

22 

NA20702 

-- 

S7 

22 

NA20901 

-- 

S7 

25 

NA21101 

S7 

11 

NA21201 

S7 

21 

NA21301 

S7 

lA 

NA21302 

— 

— 

S7 

15 

KA21A01 

— 

S7 

17 

NA21501 

— 

— 

S7 

27 

-- 

NA21601 

— 

S8 

32 

.^A 

2A 

NA21701 

— 

— 

S7 

2A 

SB 

26 

NA21801 

S7 

10 

NA21901 

S7 

16 

SA 

lA 

NA22001 

— 

S7 

26 

NA22101 

— 

S8 

16 

SI 

11 

NA22201 

S8 

19 

SI 

17 

NA22301 

— 

S8 

25 

NA22A01 

S8 

26 

S8 

27 

Hi  9  9  ^  ft  1 

S8 

2Z 

NA22601 

S8 

31 

NA22701 

S8 

20 

NA22801 

S8 

30 

NA22901 

S8 

29 

NA23001 

S8 

21 

NA23101 

S8 

17 

NA23201 

S8 

2A 

NA23202 

S8 

2A 

NA23301 

S8 

23 

NA23A01 

S8 

15 

NA23501 

S8 

18 

NA23601 

S8 

27 

NA23701 

S8 

28 

NA23801 

S9 

18 
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Table  A, 15 
(continued) 


COHORT  1 

COHORT  2 

COHORT  3 

COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

BLOCK  ITEM 

BLOCK  ITEM 

BLOCK 

ITEM 

FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

N423901 

S9 

22 

NA31201 

-- 

-- 

-- 

-- 

Sd 

lA 

N423902 

-- 

S9 

23 

NA31301 

__ 

S8 

19 

N424001 

__ 

-- 

S9 

3A 

NA31302 

__ 

__ 

S8 

19 

N424201 

__ 

-- 

S9 

36 

NA31A01 

S8 

20 

N424301 

__ 

__ 

S9 

19 

NA31501 

S8 

15 

N<;24401 

__ 

__ 

S9 

31 

NA31801 



__ 

__ 

S8 

28 

N424S01 

S9 

27 

NA31901 

__ 

__ 

__ 

S8 

17 

NA2A701 

__ 

S9 

21 

NA31902 

S8 

18 

NA2A801 

__ 

-- 

S9 

2A 

NA32001 

SI 

28 

NA2A802 

S9 

25 

NA32101 

-- 

-- 

S8 

16 

NA24803 

__ 

S9 

25 

NA32201 

S8 

23 

NA2A901 

__ 

S9 

28 

NA32301 

-- 

-- 

-- 

-- 

S8 

30 

NA25001 

__ 

__ 

S9 

20 

NA32A01 

-- 

-- 

-- 

Sd 

25 

NA25101 

__ 

-- 

S9 

35 

NA32501 

-- 

-- 

-- 

S8 

32 

NA25201 

-- 

-- 

S9 

26 

NA326C1 

-- 

-- 

-- 

S8 

22 

NA25301 

-- 

S9 

30 

NA32701 

-- 

-- 

-- 

S8 

21 

NA25A01 

S9 

33 

NA32801 

-- 

S8 

29 

NA25501 

-- 

-- 

S9 

29 

NA32901 

-- 

-- 

S8 

2A 

NA25601 

-- 

SI 

27 

NA33001 

S5 

11 

-- 

-- 

NA25701 

-- 

SI 

30 

NA33101 

S5 

18 

-- 

NA25702 

__ 

-- 

SI 

30 

NA33201 

S6 

17 

-- 

-- 

NA25801 

__ 

__ 

SI 

25 

NA33301 

S6 

5 

-- 

-- 

-- 

-- 

NA2S901 

__ 

__ 

SI 

22 

NA33A01 

S6 

8 

-- 

__ 

__ 

NA26001 

__ 

SI 

31 

NA33501 

S6 

9 

-- 

NA26101 

__ 

__ 

SI 

16 

NA33601 

SA 

20 

SA 

20 

_- 

NA26201 

__ 

^_ 

SI 

32 

NA33701 

S7 

18 

__ 

NA26A01 

__ 

__ 

SI 

20 

NA33801 

-- 

-- 

SA 

16 

NA26501 

__ 

SI 

,:s 

NA33901 

-- 

-- 

-- 

-- 

S7 

33 

NA26601 

__ 

__ 

S9 

32 

NA3A001 

_^ 

S7 

29 

NA26801 

__ 

SI 

29 

NA3A101 

-- 

-- 

-- 

S8 

33 

NA26901 

__ 

^_ 

SI 

28 

NA3A201 

-- 

-- 

SI 

33 

NA27001 

SI 

21 

NA3A202 

-- 

-- 

SI 

3A 

NA27101 

__ 

SI 

23 

NA3A301 

-- 

-- 

SA 

18 

NA27201 

__ 

-- 

SI 

18 

NA3AA01 

SA 

15 

SA 

15 

-- 

NA27202 

__ 

SI 

19 

NA3A501 

S5 

9 

-- 

-- 

-- 

-- 

NA27301 

__ 

SI 

35 

NA3A502 

S5 

9 

-- 

-- 

-- 

NA27A01 

__ 

-- 

SI 

2A 

NA3A601 

S5 

10 

-- 

-- 

NA27501 

__ 

S7 

20 

NA3A801 

-- 

-- 

-- 

SA 

15 

NA27601 

__ 

S7 

19 

NA3A901 

-- 

-- 

SA 

27 

NA27701 

-- 

S7 

35 

IIA35001 

-- 

SA 

23 

-- 

NA27801 

-- 

__ 

S7 

18 

NA35101 

-- 

S6 

16 

S6 

16 

NA27901 

-- 

__ 

S7 

32 

NA35201 

-- 

-- 

S5 

6 

S5 

6 

NA28001 

-- 

S7 

25 

NA35301 

-- 

-- 

S9 

37 

NA28101 

-- 

__ 

S7 

21 

NA35A01 

-- 

S9 

lA 

SI 

10 

NA2dl02 

-- 

-- 

S7 

22 

NA35501 

-- 

S9 

15 

-- 

-- 

NA28201 

__ 

-- 

S7 

26 

NA35601 

-- 

-- 

39 

20 

NA28202 

S7 

26 

NA35701 

-- 

S9 

21 

NA28301 

-- 

S> 

27 

NA35801 

-- 

-- 

S9 

22 

SI 

20 

NA28A01 

-- 

S7 

3A 

NA35901 

-- 

-- 

S9 

23 

SI 

23 

NA28501 

-- 

S7 

23 

NA36001 

-- 

S9 

2A 

-- 

NA28601 

-- 

-- 

S7 

30 

NA36107 

-- 

-- 

S9 

25 

-- 

NA28801 

-- 

S7 

28 

NA36201 

-- 

-- 

S9 

26 

SI 

2A 

NA28901 

-~ 

S7 

2A 

NA36301 

-- 

S9 

27 

NA29001 

-- 

-- 

S7 

37 

NA36A01 

-- 

S9 

28 

SI 

22 

NA29101 

-- 

S7 

36 

NA36501 

-- 

S9 

29 

SI 

21 

NA29201 

-- 

S7 

31 

NA36601 

S9 

31 

NA29A01 

-- 

SA 

19 

NA36701 

S9 

17 

NA29601 

-- 

-- 

SA 

25 

NA36801 

-- 

-- 

-- 

SI 

13 

K429701 

-  - 

SA 

29 

NA36802 

SI 

lA 

NA29801 

SA 

27 

NA36901 

SI 

15 

NA29901 

SA 

13 

NA37001 

SI 

16 

NA30001 

SA 

20 

NA37002 

SI 

16 

NA30002 

S4 

21 

NA37101 

SI 

17 

NA300Q3 

SA 

22 

NA37201 

SI 

18 

NA30101 

SA 

23 

NA37202 

SI 

19 

NA30301 

SA 

30 

NA37301 

SI 

25 

NA30A01 

SA 

12 

NA37A01 

SI 

27 

NA30501 

SA 

26 

NA37501 

SI 

29 

NA30601 

SA 

28 

NA37601 

S7 

8 

NA30801 

SA 

17 

NA37701 

S7 

9 

NA30802 

SA 

17 

NA37801 

S7 

10 

NA30901 

SA 

31 

NA37901 

S7 

11 

NA31101 

se 

31 

NA38001 

S7 

12 
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COHORT  1 


FIELD 

BLOCK 

ITEM 

N438101 

S7 

13 

N438201 

S7 

lA 

N438301 

S7 

15 

NA38A01 

S7 

16 

NA38501 

S7 

17 

NA38601 

S7 

18 

N438701 

S7 

19 

NA38801 

S7 

20 

N438901 

S7 

21 

Table  A, 15 
(continued) 


COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


-AA3- 


Table  A. 16 
COMPUTER  COMPETENCE  COGNITIVE  ITEMS 


COHORT  1 

COHORT  2 

COHORT  3 

COHORT  1 

COHORT  2 

COHORT  3 

PTFT  n 

BLOCK 

ITEM 

BLOCK 

ITEM 

FIELD 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

M AH  n 1 n 1 
nou  u lu 1 

CI 

20 

C 1 

Z  X 

L>  X 

21 

N603706 

C3 

10 

C3 

10 

nouui 

CI 

21 

CI 

22 

u  X 

z^ 

N603707 

C3 

11 

C3 

11 

noUU 1 U  J 

CI 

22 

CI 

23 

r*  1 
u  X 

Z  J 

N603708 

C3 

12 

C3 

12 

M  A  n  n  1  n  A 

CI 

23 

CI 

24 

U  X 

9  A 
Z*4 

N603801 

CA 

20 

noUU lU  D 

CI 

24 

C 1 

25 

U  X 

^  J 

N603901 

CA 

21 



__ 

MAnn 1  HA 
nouu 1 uo 

CI 

25 

C 1 

9  A 

zb 

r*  1 
u  X 

0  A 
^O 

N60A001 

CA 

22 

u Ann o  m 

C2 

lA 

C2 

lA 

C2 

1 A 

N60ACC2 

CA 

23 

Men  ft  "3  n  1 
NoUU J  U 1 

C2 

1 S 

C2 

15 

UZ 

1  K 

X  J 

N60A003 

CA 

2A 





C2 

16 

C2 

16 

UZ 

1  A 
X  O 

N60A 101 

CA 

30 

U  A  ft  ft  ^  ft  1 

C2 

17 

C2 

17 

C2 

1  / 

N60A201 

CA 

2A 

u  c  /\  n  c  ft  1 

C2 

18 

C2 

18 

C2 

18 

N60A202 

CA 

25 

UC      A  T  ft  1 

NdOU/ 0 1 

C2 

19 

C2 

lo 

N60A203 

C  A 

26 

%iA nft  Q  ft  1 

C3 

12 

C  A 

A 

o 

N60A301 

C5 

13 

UA  nno  n i 

C3 

13 

CA 

/ 

N60AA01 

C5 

1 A 

C5 

29 

U  A  ft  ft  Q  ft  o 

C3 

lA 

CA 

Q 

o 

N60AA02 

C5 

15 

C5 

30 

UA  ft 1  ft  ft 1 

C3 

1 S 

CA 

Q 

N60A501 

C5 

16 

C5 

31 

n  o  u  X  X  u  X 

C3 

16 

C  A 

1  ft 

xu 

N60A601 

C6 

16 

C6 

2A 

n  o  u  x^  u  X 

C3 

1 

U  J 

X 

N60A602 

C6 

17 

C6 

25 

U  A  ft  1  1  ft  1 
MDU  X J  U  X 

C3 

2 

r"3 

Z 

N60A701 

C6 

18 

C6 

18 

u  A  n  1  A  n  1 
n  o  u  x*t  u  X 

r*  T 
Lo 

J 

u  J 

•a 
J 

N60A801 

C6 

19 

HAH 1 

n  o  u  X  D  u  X 

C  T 
U  J 

A 

u  J 

N60A901 

CI 

32 

ci 

33 

CI 

33 

u  A  n  1  A  n  1 
n  o  u  xo  u  X 

C3 

13 

r*  T 
L  J 

1  T 
X  J 

N605001 

CI 

3  3 

CI 

3  A 

CI 

3A 

H  A  ft  1  A  ft  O 

nbU lb  UZ 

C3 

lA 

U  J 

1  A 
X  H 

N605101 

CI 

3  A 

C 1 

3  5 

CI 

35 

U  A  ft  1  A  ft  Q 

Mb  u  Xbu J 

U  J 

X  J 

C3 

1 5 

N605201 

CI 

3  5 

CI 

36 

CI 

36 

iiAft  1  7  n  1 
n  D  V  X  /  u  X 

r*  A 
u*i 

X  X 

rs 

19 

N605301 

CI 

36 

CI 

37 

CI 

37 

MA  ft  1  7  ft  9 

C  A 

1  o 

X£. 

20 

N605302 

CI 

36 

CI 

37 

CI 

37 

N  A  n  1  7  0*^ 

C  A 

1  T 
X  J 

21 

N605A01 

C2 

26 

C2 

27 

Nfi  01  7CiL 

C  A 

1  A 

C5 

22 

N605501 

C2 

27 

C2 

28 





MA  n 17ns 

C  A 

15 

C5 

23 

N605601 

C2 

28 

C2 

29 



MAfti 7nR 

1  A 
XO 

C5 

2A 

N605602 

C2 

29 

C2 

30 





UA  fti 7  07 
n  o  u  X  /  u  / 

C  A 

C5 

25 

N605701 

C2 

30 

C2 

31 





N601708 

CA 

18 

C5 

26 

N605702 

C2 

30 

C2 

31 

N601709 

CA 

19 

C5 

27 

N605801 

C2 

27 

N601801 

CA 

22 

N605901 







C2 

28 

K601901 

CA 

23 

N606001 

__ 



C2 

29 

N6020G1 

CA 

29 

N606101 







C2 

30 

N602101 

CA 

27 

N606102 

__ 

__ 



C2 

31 

N60??01 

11  V         ^  V  X 

C6 

15 

C  A 

28 

N606103 







C2 

32 

£l  V  V  ^      V  X 

N60610A 

__ 

C2 

30 

MAftO  Afti 
n    yjCtt  \i  X 

C5 

g 

N606106 

__ 

__ 

C2 

31 

g 

N606108 





C2 

32 

£1  V  V  ^  V  V  JL 

1 '? 

x£. 

C  5 

28 

N606201 

C3 

2A 

__ 

M Aft07ni 

17 

C5 

3  2 

N606301 

C3 

25 



_ 

MAftonni 
n  u  u    o  u  X 

C6 

1 1 

N606A01 

C3 

26 

_  _ 

_  _ 

U Aft?  ftO? 

il  U  U  £■  O  U 

r*  A 

12 

N606501 

C3 

27 

_  ^ 



M  Aft?  RO'^ 
n  V  u    o  u 

r*  A 

X  J 

N606601 

C3 

28 



_  _ 

_  _ 

N  AftO  flftA 

nA 

1  A 
X*4 

N606701 

C  3 

20 

C3 

20 

n  V  u  ^  9  V  X 

C6 

20 

N606702 

__ 

C3 

21 

C3 

21 

M AftTOOl 
n  \j  \j  >j  \j  \j  X 

C6 

19 

N606703 

C3 

22 

C3 

22 

N603002 

C6 

20 

N60670A 

C3 

23 

C3 

23 

N603003 

C6 

21 

N606801 



C3 

2A 

C3 

2A 

N60300A 

C6 

22 

N606901 



CA 

27 

N603005 

C6 

23 

N607001 

CA 

28 



-- 

N603101 

CI 

19 

CI 

20 

CI 

20 

N607101 

__ 



CA 

29 

-- 

N603201 

CI 

26 

CI 

27 

CI 

27 

N607102 





CA 

30 

__ 

N603202 

CI 

27 

CI 

28 

CI 

28 

N607201 





CA 

31 

N603203 

CI 

28 

CI 

29 

CI 

29 

N607301 



^  _ 

CA 

33 

N60320A 

r*  1 

U  X 

29 

CI 

3  0 

CI 

30 

N607302 

__ 





CA 

3A 

£l  W           V  X 

1  Q 
X  d 

C2 

20 

C2 

20 

N607303 



CA 

35 

N603302 

C2 

20 

C2 

21 

C2 

21 

N6C730A 

CA 

36 

N603303 

C2 

21 

C2 

22 

C2 

22 

N607305 

CA 

37 

N603401 

C2 

22 

C2 

23 

C2 

23 

N607306 

CA 

38 

N603A02 

C2 

23 

C2 

2A 

C2 

2A 

N607307 

CA 

39 

N603501 

C3 

17 

N607A01 

C5 

20 

N603502 

C3 

18 

N607501 

C5 

21 

N603503 

C3 

19 

N607601 

C5 

22 

N60350A 

C3 

20 

N607602 

05 

23 

N603601 

C3 

21 

N607603 

C5 

2A 

N603701 

C3 

5 

C3 

5 

N607701 

C5 

35 

N603702 

C3 

6 

C3 

6 

N607702 

C5 

36 

N603703 

C3 

7 

C3 

7 

N607703 

C5 

37 

N60370A 

C3 

8 

C3 

8 

N607801 

C5 

38 

N603705 

C3 

9 

C3 

9 

N607901 

06 

23 
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Table  A, 16 
(continued) 


COHORT  1  COHORT  2  COHORT  3 


FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

N606001 

**  ~* 

•**• 

C6 

2A 

•*•* 

N606002 

**  ** 

•*  ~ 

C6 

25 

N606101 

•*  •* 

— 

C6 

26 

N606103 

~  ~ 

C6 

28 

N606201 

C6 

28 

N606301 

-  - 

— 

C6 

29 

N606302 

C6 

30 

N608303 

~  - 

— 

C6 

31 

N608401 

~  — 

C6 

32 

N608A02 

"■*• 

~  - 

— 

C6 

32 

N608S01 

CI 

37 

CI 

38 

-- 

N60860 1 

CI 

38 

CI 

39 

-- 

N60870 1 

CI 

39 

CI 

40 

-- 

N608702 

CI 

40 

CI 

41 

-- 

N60880 1 

"** 

~  ~ 

CI 

42 

N60890 1 

C2 

31 

C2 

32 

N609001 

C2 

32 

C2 

33 

N609101 

C2 

33 

C2 

341 

-- 

N609201 

C2 

3A 

C2 

35 

N609202 

C2 

34 

C2 

35 

-- 

-- 

N609301 

C3 

29 

-~ 

-- 

H609A01 

C3 

30 

— 

-- 

-- 

N60950 1 

C3 

31 

-- 

-- 

-- 

N60960 1 

C3 

32 

-- 

-- 

N609602 

C3 

33 

-- 

N609603 

C3 

33 

— 

-- 

N609701 

~  ~ 

C3 

25 

N60960 1 

~~ 

C3 

26 

-- 

Nd0990 1 

C3 

27 

-- 

ff  DlOOO  1 

C3 

28 

— 

-- 

NblOlO 1 

C4 

32 

— 

UP1 n 1  no 
n  D  X  u 1 

C4 

33 

— 

n  D  X  u  xu  o 

C4 

34 

— 

n  o  X  U  X 

C4 

35 

n  D  X  U  OU  X 

C^ 

36 

N61 n&n 1 

n  u  X  u  *t  u  X 

C4 

37 

n  Dxuou  X 

C5 

25 

— 

nblObO 1 

C5 

26 

-- 

UC  1  n  ^  /\  4 

Nd10/0 1 

C5 

27 

-- 

n  D  X  U  /  U  Z 

C5 

28 

— 

n  D  X  u  /  u  o 

C5 

29 

— 

£1  D  X  U  /  U  •* 

C5 

30 

— 

N61080 1 

C6 

29 

-- 

N610802 

C6 

30 

-- 

n  bluouo 

C6 

31 

MAI n  on  1 

C6 

32 

N611001 

C6 

33 

-- 

M  blliu 1 

— 

CI 

38 

Nblll03 

— 

— 

CI 

40 

no  1 xZU  X 

CI 

41 

MR  1 1 ono 

CI 

42 

MR  1  1  o  n 1 
Mb  1 XZUo 

CI 

43 

H  R  1  1  o  n  X 

CI 

44 

MR  1  1     n  1 

Mb 1 ioU 1 

— 

C2 

33 

~  - 

C2 

34 

MR  1  1  '>f\'> 

— 

C2 

35 

ftic  1  1     /^  1. 

nbxl304 

**  ~ 

— 

C2 

36 

N61 140 1 

— 

— 

C3 

25 

N611402 

— 

C3 

26 

N611403 

27 

N611A04 

C3 

28 

N611501 

C4 

40 

N611601 

C5 

39 

N611602 

C5 

40 

N611603 

C5 

41 

N611604 

C5 

42 

N611605 

C5 

43 

N611606 

C5 

44 

N611701 

C6 

33 

N611702 

C6 

34 

N611703 

C6 

35 

N611801 

C6 

36 
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Table  A. 17 
U.S.  HISTORY  COGNITIVE  ITEMS 


FIELD 

HOOOlOl 

H000201 

H000301 

HOOOAOl 

HOOOSOl 

H000601 

H000701 

HOOOSOl 

H000901 

HOOlOOl 

HOOllOl 

H001201 

H001202 

H001203 

H00120A 

H001205 

H001301 

HOOlAOl 

H001501 

H001601 

H001701 

H001801 

H001901 

H002001 

H002101 

H002201 

H002301 

H002401 

H002402 

H002403 

H002404 

H002A05 

H002A06 

H002A07 

H002A08 

H002501 

H002601 

H002701 

H002801 

H002901 

H003001 

H003101 

H003201 

H003301 

H003401 

H003501 

H003601 

H003701 

H003801 

H003901 

H004001 

B004101 

H004201 

H004301 

H004401 

H00A501 

H00A502 

H004601 

H00A701 

H004801 

H00A901 

H005001 

H00500A 

H005005 

H005006 

H005007 

H005008 

H005009 

H005010 

H005101 

H005102 

E005103 

H005201 


COHORT  1 
BLOCK  ITEM 


COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

HI 

H2 

H2 

H2 

H2 

HI 

HI 

HI 

HI 

HI 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H2 

H4 

H4 

H4 

H4 

H4 

H4 

H4 

H2 

H2 

H2 

H2 


13 

1<> 

15 

16 

17 

18 

19 

20 

21 

22 

23 

2A 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

AO 

Al 

A2 

A3 

20 

21 

22 

23 

AA 

A5 

A6 

A7 

A8 

13 

lA 

15 

16 

17 

18 

19 

2A 

25 

26 

27 

28 

29 

30 

31 

32 

33 

3A 

35 

36 

37 

38 

17 

18 

19 

20 

21 

22 

23 

39 

AO 

Al 

A2 


FIELD 

H005301 

H005A01 

H005501 

H005601 

H005701 

H005801 

H005901 

H006001 

H006101 

H006201 

H006301 

H006A01 

H006501 

H006601 

H006701 

H006801 

H006901 

H007001 

H007101 

H007102 

H007103 

H007201 

H007301 

H007A01 

H007501 

H007601 

H007701 

H007801 

H007901 

H008001 

H006101 

H008201 

H008301 

H008302 

H008303 

H00830A 

H008305 

H008A01 

H008501 

H008601 

H008701 

H008801 

H008901 

H009001 

H009101 

H009201 

H009301 

H009A01 

H009501 

H009601 

H009701 

H009801 

H009901 

HOlOOOl 

HOIOIOI 

H010201 

H010301 

HOlOAOl 

H010501 

H010601 

H010701 

H010801 

H010901 

HOllOOl 

HOlllOl 

H011201 

H011301 

HOllAOl 


COHORT  1 
BLOCK  ITEM 


COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


H2 

H2 

H2 

H2 

H2 

H2 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

^3 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

EA 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

HA 

BA 

HA 

HA 

HA 

HA 

HA 

H4 


A3 

AA 

A5 

A6 

A7 

A8 

13 

lA 

15 

16 

17 

18 

19 

20 

21 

22 

23 

2A 

25 

26 

27 

28 

29 

30 

31 

32 

33 

3A 

35 

36 

37 

38 

39 

AO 

Al 

A2 

A3 

AA 

A5 

A6 

A7 

13 

lA 

15 

16 

2A 

25 

26 

27 

28 

29 

30 

31 

32 

33 

3A 

35 

36 

37 

38 

39 

AO 

Al 

A2 

A3 

AA 

A5 

A6 


-AA6- 
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Table  A. 18 
LITERATURE  COGNITIVE  ITEMS 


FIELD 

LOOOlOl 
L000201 
L000301 
LOOOAOl 
L000501 
L000601 
L000701 
L000801 
L000901 
LOOlOOl 
LOOllOl 
L001201 
L001301 
LOOlAOl 
L001501 
L001601 
L001701 
L001801 
L001901 
L002001 
L002101 
L002201 
L002301 
L002401 
L002501 
L002601 
L002701 
L002801 
L002901 
L003001 
L003101 
L003201 
L003301 
L003A01 
L003501 
L003601 
L003701 
L003801 
L003901 
LOOAOOl 
LOOAlOl 
L00A201 
L00A301 
LOOAAOl 
L00A501 
L00A601 
L00A701 
L00A801 
L00A901 
L005001 
L005101 
L005201 
L005301 
L005A01 
L005501 
L005601 
L005701 
L005801 
L005901 
L006001 
L006101 
L006201 
L006301 
L006A01 
L006501 
L006601 
L006701 
L006801 
L006901 
L007001 
L007101 
L007201 
L007301 


COHORT  1 
BLOCK  ITEM 


COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
LI 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L2 
L3 
L3 
L3 
L3 
L3 
L3 
L3 
L3 
L3 
L3 
L3 
L3 


19 
20 
21 
22 
23 
2A 
25 
26 
27 
28 
29 
30 
31 
32 
33 
3A 
35 
36 
37 
38 
39 
AO 
Al 
A2 
A3 
AA 
A5 
A6 
A7 
A8 
19 
20 
21 
22 
23 
2A 
25 
26 
27 
28 
29 
30 
31 
32 
33 
3A 
35 
36 
37 
38 
39 
AO 
Al 
A2 
A3 
AA 
A5 
A6 
A7 
A8 
A9 
19 
20 
21 
22 
23 
2A 
25 
26 
27 
28 
29 
30 


FIELD 

L007A01 
L007501 
L007601 
L007701 
L007801 
L007901 
L008001 
L008101 
L008201 
L008301 
L008A01 
L008501 
L008601 
L008701 
L008801 
L008901 
L009001 
L009101 
L009201 
L009301 
L009A01 
L009501 
L009601 
L009701 
L009801 
L009901 
LOlOOOl 
LOIOIOI 
L010201 
L010301 
LOlOAOl 
L010501 
L010601 
L010701 
L010801 
L010901 
LOllOOl 
LOlllOl 
L011201 
L011301 
LOllAOl 
L011501 
L011601 
L011701 
L011801 
L011901 
L012001 
L012101 


COHORT  1 


COHORT  2 


BLOCK 

ITEM 

BLOCK 

ITEM 

_  _ 

__ 

L3 

3 1 

^  _ 

_  _ 

L3 

32 

_ 

L3 

33 

__ 

L3 

3  A 

__ 

L3 

35 

L3 

36 

L3 

37 

L3 

38 

L3 

39 

L3 

AO 

^_ 

L3 

A  1 



T  "a 
L  J 

A2 

__ 

L3 

A  3 

L3 

A  A 

L3 

A5 

L3 

A6 

L3 

A7 

L3 

A8 

LA 

19 

LA 

20 

LA 

21 

__ 

LA 

22 

LA 

23 



__ 

T  /. 
L.H 

2A 





T  J. 

25 



T  /. 

26 



T  /. 

27 

_  ^ 

T  L 

*>  fi 
zo 

__ 



T  A 



__ 

T  /. 

30 

T  /. 

31 



T  A 

•a  *> 

__ 



LA 

J  J 

__ 

-  _ 

T  J. 

3  A 



LA 

O  J 

LA 

36 

LA 

37 

LA 

38 

LA 

39 

LA 

AO 

LA 

Al 

LA 

A2 

LA 

A3 

LA 

AA 

LA 

A5 

LA 

A6 

LA 

A7 

LA 

A8 

-««7- 
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Table  A. 19 
COMMON  BACKGROUND  ITEMS 


COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

RT  npv 

X  xjdn 

Ti  f\  fx  fx  f\  fx  \ 

BO  00901 

Bl 

/ 

D  X 

y 

Bl 

7 

CU  UU9UZ 

Bl 

6 

ol 

0 

Bl 

3 

B  A  A  A  n 

o00090j 

Bl 

9 

ol 

Q 

Bl 

g 

D  1 

1  ft 
xu 

Bl 

10 

Bl 

10 

Bl 

11 

R  1 
D  X 

X  X 

Bl 

11 

OU  U  JLoU  JL 

Bl 

12 

D  1 

xz 

Bl 

12 

"*  n  A  ■a  n  n  1 

Bl 

1 

D  1 

X 

Bl 

\ 

OU  U J lUl 

Bl 

2 

R1 
D  X 

2 

Bl 

2 

p  n  ft  "5  o  ft  1 

Bl 

3 

oU  U<3  1 

Bl 

3 

Bl 

3 

SU  U  <3  *l  U  X 

Bl 

A 

D  X 

A 

Bl 

000*5  <ft1 
OUUJ  DUX 

Bl 

5 

Bl 

5 

Bl 

5 

Q  n  ft*)  R  ft  1 

Bl 

6 

R  1 
D  X 

D 

Bl 

5 

3;  ,s  ft  "J  T  ft  1 

Bl 

13 

R  1 
Ol 

X  J 

B 1 

13 

D  A  A  ■a  O  A  1 

Bl 

lA 

Q  A  A         A  1 

R  1 
D  1 

1  L 

B 1 

1 A 

S  A  A  /  A  A  1 

Bl 

15 

Bl 

1 D 

R  1 

a  X 

X  D 

^  f\  fx  i.  ^  A  ^ 

BOOA 10 1 

El 

16 

B00A201 

Bl 

16 

B 1 

17 

B 1 

1  / 

B00A301 

Bl 

IQ 

Bl 

16 

R  1 

OX 

XO 

BOOA AOl 

Bl 

20 

Bl 

19 

R  1 
D  1 

X  «3 

BOOASOl 

Bl 

21 

Bl 

20 

Tt  f\f\  M   C  fx  A, 

BOOAbOl 

Bl 

22 

Bl 

21 

B00A701 

Bl 

23 

~" 

BOOA  601 

Bl 

22 

B00A901 

Bl 

23 

B005001 

B 1 

20 

Q  nft  ^  1  ft  1 

B 1 

21 

n  Aft  ^  Oft  1 
dUO^ZO 1 

Bl 

22 

IJftft  ^Oft  o 
DUU  3ZUZ 

B 1 

23 

n  ftft  ^  Oft  ^ 

Bl 

2A 

Qftft 

Bl 

25 

a\J\J  J  0\J  X. 

Bl 

26 

D\f  \f  J  >J  \J  ^ 

Bl 

27 

n  ft  ft  ^  *3  ft 

B 1 

26 

ft  ft  ^    ft  A 

Bl 

29 

n  ft  ft  ^  *3  ft  ^ 

Bl 

30 

D  U  U  J  J  U  D 

Bl 

3 1 

O  ft  ft  ^  1ft  ^ 
OUU3 JU / 

Bl 

32 

Qftft  S'^OR 

Bl 

33 

nftft  ^"^fto 

Bl 

3A 

6005310 

Bl 

35 

nftft      1 1 
DU U ^ J  XX 

Bl 

36 

n  ft  ft  c  q  1  o 
DUU  ^ J  XZ 

Bl 

37 

B005313 

Bl 

36 

6005A01 

Bl 

2A 

Bl 

39 

600  5501 

Bl 

A2 

6005601 

Bl 

2A 

Bl 

26 

Bl 

A3 

B005701 

Bl 

25 

Bl 

27 

Bl 

AA 

B005801 

Bl 

26 

Bl 

28 

Bl 

A5 

6005901 

Bl 

27 

6006001 

Bl 

29 

Bl 

A6 

6006101 

Bl 

26 

6006201 

Bl 

30 

Bl 

A7 

B00630I 

Bl 

A6 

B0C6302 

Bl 

A6 

B006501 

Bl 

Al 

i. 
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Table  A. 20 
READIHG  BACK6R0UHD  ITEMS 


COHORT  1  COHORT  2  COHORT  3 


FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

SOO 1001 

R2 

7 

R2 

7 

R2 

7 

SOO 1002 

R2 

8 

R2 

8 

R2 

8 

S001003 

R2 

9 

R2 

9 

R2 

9 

S002701 

Bl 

17 

Bl 

16 

Bl 

16 

S003301 

R3 

1 

Rl 

19 

Rl 

19 

S003501 

R3 

2 

— 

~  - 

S003502 

R3 

3 

— 

— 

S003503 

R3 

A 

— 

— 

S003S04 

R3 

5 

~  " 

*■  ~ 

— 

— 

O  A  A  O  C  C 

S003505 

R3 

6 

— 

— 

C»  A  A  ^  C  A  C 

8003506 

R3 

7 

_  — 

— 

— 

S004001 

—  ^ 

Bl 

25 

Bl 

AO 

O  A  A  /  ^  A  ^ 

S00A301 

R3 

1 

R3 

1 

O  A  A  /.  ^  A  O 

S004302 

R3 

2 

R3 

2 

O  A  A  /  ^  A  ^ 

S00*»303 

R3 

3 

R3 

3 

O  A  A  /  «^  A  1_ 

S00430A 

R3 

A 

R3 

A 

SOO A305 

R3 

5 

R3 

5 

S004306 

—  — 

R3 

6 

R3 

6 

S00<»307 

R3 

7 

R3 

7 

S004308 

R3 

8 

R3 

8 

O  A  A  /  ^  A  A 

SOO  s309 

R3 

9 

R3 

9 

SOO A310 

R3 

10 

R3 

10 

O  A  A  /  ^  ^  ^ 

SOO A311 

R3 

11 

R3 

11 

SOOA AOl 

Rl 

10 

— 

— 

— 

SOO A A02 

Rl 

11 

— 

— 

S004i501 

~~ 

R3 

12 

R3 

12 

C*  A  A  i.  f  A  ^ 

SOO  ^502 

R3 

13 

R3 

13 

O      rt  /.  C  A  1 

R3 

lA 

R3 

lA 

o  n  n  i.  C  A  i. 

bUU  *i  _)04 

R3 

15 

R3 

15 

bUU<»DO  1 

R2 

1 

R2 

1 

R2 

1 

A  A  L  C  f\  ^ 

SOO  <i  602 

R2 

2 

R2 

2 

R2 

2 

O  A  A  /  C  A  '3 

bOO  s  oOo 

R2 

3 

R2 

3 

R2 

3 

O  A  A  JL  ^  A  ^ 

b004/01 

R2 

A 

R2 

A 

R2 

A 

RA 

1 

RA 

1 

SOO A702 

R2 

5 

R2 

5 

R2 

5 

RA 

2 

RA 

2 

O  A  A  /  T  A  O 

SOU*i  703 

R2 

6 

R2 

6 

R2 

6 

O  A  A  /  T  A  /. 

bOO*i  /Oh 

RA 

3 

RA 

3 

O  A  A  J.  ^  A  f 

RA 

A 

RA 

A 

Q  A  A  A  T  A  C 

RA 

5 

RA 

5 

Q  A  A  A  7  A  7 
bUU  4  /  U  / 

RA 

6 

RA 

6 

~7 

RA 

7 

RA 

7 

Rl 

1 

Rl 

1 

Rl 

1 

O  U  U  J  1  u  z 

B 1 

Al 

2 

Rl 

2 

Rl 

2 

OU  U  J  X  u  o 

B  1 
Al 

3 

Rl 

3 

Rl 

3 

OU  U  J  X  u  *t 

Al 

A 

Rl 

A 

Rl 

A 

13  1 
Al 

c 
J 

Rl 

5 

Rl 

5 

w  V  V  ^  X  V  w 

Rl 

c 
O 

Rl 

6 

Rl 

6 

Al 

7 

Rl 

7 

Rl 

7 

O  U  U  J  z  u  z 

Al 

8 

Rl 

8 

Rl 

8 

Q  n  n  s  9  n  "3 

Rl 

9 

Rl 

9 

Rl 

9 

O  U  U  J  O  U  1 

Rl 

10 

Rl 

10 

bUU  jOUZ 

Rl 

11 

Rl 

11 

bUU  joUo 

Rl 

12 

Rl 

12 

Q  A  A  ^  "3  A  /. 

bUU  5o 0*1 

Rl 

13 

Rl 

13 

^  A  A  C  "5  A  C 

Rl 

lA 

Rl 

lA 

O  A  A  C  /.  A  4 

Rl 

15 

Rl 

15 

O  A  A  f  1_  A  A 

5005^02 

Rl 

16 

Rl 

16 

O  A  A  C  /  A  ^ 

S0054  03 

Rl 

17 

Rl 

17 

O  A  A  C  /  A  / 

bOO  5s  \JH 

Rl 

18 

Rl 

18 

SOO  690 1 

RA 

H 

AO 

A 

R6 

A 

S007001 

RA 

3 

R6 

3 

R6 

3 

S007002 

RA 

3 

R6 

3 

R6 

3 

S007003 

RA 

3 

R6 

3 

R6 

3 

S00700<» 

RA 

3 

R6 

3 

R6 

3 

S007005 

RA 

3 

R6 

3 

R6 

3 

S007006 

RA 

3 

R6 

3 

R6 

3 

S007007 

RA 

3 

R6 

3 

R6 

3 

S007008 

RA 

3 

R6 

3 

R6 

3 

S007009 

RA 

3 

R6 

3 

R6 

3 

S007010 

RA 

3 

R6 

3 

R6 

3 

S007201 

RA 

1 

R6 

1 

R6 

1 

S007202 

RA 

2 

R6 

2 

R6 

2 

COHORT  1  COHORT  2  COHORT  3 


FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

S007301 

R5 

1 

S007302 

R5 

2 

S007303 

R5 

3 

S00730A 

R5 

A 

-- 

S007305 

R5 

5 

S007306 

R5 

6 

S007307 

R5 

7 

— 

S007308 

R5 

8 

S007309 

R5 

9 

S007310 

R5 

10 

S007A01 

R6 

1 

S007A02 

R6 

2 

S007A03 

R6 

3 

S007A0A 

R6 

A 

S007901 

R5 

1 

R5 

S008001 

R5 

2 

R5 

S008101 

R5 

3 

R5 

S008201 

R5 

A 

R5 

S008301 

R5 

5 

R5 

S008A01 

R5 

6 

R5 

-AA9- 


Table  A. 21 
MATHEMATICS  BACKGROUND  ITEMS 


COHORT  1 

FIELD 

BLOCK 

ITEM 

S200901 

M3 

1 

S200902 

M3 

2 

S200903 

M3 

3 

S201601 

S201602 



S201603 



S20160A 





S201605 

S201606 



S201607 





S201608 





S201609 

__ 

S201610 

S201611 

__ 

S201612 

S201613 

S20161A 

S202201 



S202202 





S202203 

__ 

S20220A 

S202205 

52^2206 

__ 

S2U2207 

__ 

S202901 

S202902 





S202903 





S20290A 





S202905 





S202906 





S202907 



S202908 

__ 

S202909 

__ 

__ 

S202910 

S202911 



S203701 

__ 

S203702 

S203703 

__ 

__ 

S20370A 

-_ 

S203705 

-- 

S2G3706 

-- 

S203707 

-- 

S203708 

-- 

S203709 

-- 

S203710 

-- 

S203711 

-- 

S203712 

-- 

S203713 

-- 

S20371A 

-- 

S205A01 

S205A02 

-- 

-- 

S205A03 

-- 

S205A0A 

-- 

S205A05 

S205A06 

S205501 

MA 

1 

S205502 

MA 

2 

S205503 

MA 

3 

S20550A 

MA 

A 

S205601 

MA 

5 

S205701 

MA 

6 

S205801 

MA 

7 

S205901 

M5 

1 

S205902 

M5 

2 

S205903 

M5 

3 

S20590A 

M5 

A 

S205905 

M5 

5 

S205906 

M5 

6 

S206001 

S206101 

M5 

7 

S206102 

M5 

8 

S206103 

M5 

9 

S20610A 

M5 

10 

ERIC 


COHORT  2  COHORT  3 

BLOCK     ITEM  BLOCK  ITEM 


Ml 

1 

Ml 

1 

Ml 

2 

Ml 

2 

Ml 

3 

Ml 

3 

Ml 

A 

Ml 

A 

Ml 

5 

Ml 

5 

Ml 

6 

Ml 

6 

Ml 

7 

Ml 

7 

Ml 

8 

Ml 

8 

Ml 

9 

Ml 

9 

Ml 

10 

Ml 

10 

Ml 

11 

Ml 

11 

Ml 

12 

Ml 

12 

Ml 

13 

Ml 

13 

Ml 

lA 

Ml 

lA 

M2 

1 

-- 

M2 

2 

M2 

3 

M2 

A 

-- 

M2 

5 

M2 

6 

M2 

7 

M3 

1 

M3 

1 

M3 

2 

M3 

2 

M3 

3 

M3 

3 

M3 

A 

M3 

A 

M3 

5 

M3 

5 

M3 

6 

M3 

6 

M3 

7 

M3 

7 

M3 

8 

M3 

8 

-- 

-- 

M3 

9 

-- 

M3 

10 

-- 

-- 

M3 

11 

-- 

-- 

M2 

1 

-- 

-- 

M2 

2 

-- 

-- 

M2 

3 

-- 

-- 

M2 

A 

-- 

M2 

5 

M2 

6 

_- 

M2 

7 

M2 

8 

M2 

9 

M2 

10 

M2 

11 

M2 

12 

M2 

13 

M2 

lA 

MA 

1 

MA 

1 

MA 

2 

MA 

2 

MA 

3 

MA 

3 

MA 

A 

MA 

A 

MA 

5 

MA 

5 

MA 

6 

MA 

6 

MA 

7 

MA 

7 

MA 

8 

MA 

8 

MA 

9 

MA 

9 

MA 

10 

MA 

10 

M5 

15 

M5 

15 

M5 

16 

M5 

16 

M5 

1 

M5 

1 

M5 

2 

M5 

2 

M5 

3 

M5 

3 

M5 

A 

M5 

A 

COHORT  1 


FIELD 

BLOCK 

ITEM 

S206601 

S206701 

M7 

1 

S206702 

M7 

2 

S206703 

M7 

3 

S20670A 

M7 

A 

S206705 

M7 

5 

S206706 

M7 

6 

S207101 

M7 

7 

S207102 

M7 

8 

S207103 

M7 

9 

S20710A 

M7 

10 

3207201 

M5 

11 

S207501 

-- 

S207502 

-- 

S207503 

-- 

S20750A 

-- 

-- 

S207505 

S207506 

-- 

S207507 

-- 

S207508 

-- 

S207509 

-- 

S207510 

-- 

S207511 

S207512 

S207601 

M6 

1 

S20/eQ2 

M6 

2 

S207603 

M6 

3 

S20760A 

M6 

A 

S207605 

M6 

5 

S207606 

M6 

6 

S207607 

M6 

7 

c  o  n  T  c  n  Q 

M6 

8 

S208501 

C*      n  O  C  A  1 

S20oo0 1 

S208701 

-- 

c  o  n  Q  Q  n  1 

S208802 

S208803 

S20880A 

S208805 

S208806 

S208901 

S208902 

S208903 

S20890A 

S209001 

S209501 
S209502 
S209503 
S20950A 
S209505 
S209506 
S209507 
S209508 
S209509 
S209510 
S209511 
S209512 
S209513 
S20951A 
S209515 
S209516 
S211001 
S211002 
S211003 
S21100A 
S211005 
S211006 
S211007 

A50- 


504 


COHORT  2  COHORT  3 

BLOCK     ITEM  BLOCK  ITEM 


MA 

-- 
-- 

11 

MA 

-- 

11 

-- 
-- 

M6 

1 

M6 

1 

M6 

2 

M6 

2 

M6 

3 

M6 

3 

M6 

A 

M6 

A 

M6 

5 

M6 

5 

M6 

6 

M6 

6 

M6 

7 

M6 

7 

M6 

8 

M6 

8 

M6 

9 

M6 

9 

M6 

10 

M6 

10 

M6 

11 

M6 

-- 
-- 
-- 
-- 

12 

-- 

-- 

-- 

-- 
-- 

-- 
-- 
-- 
-- 
-- 

-- 
-- 

HA 

-- 
-- 

12 

MA 

-- 
12 

MA 

13 

MA 

13 

MA 

lA 

MA 

lA 

M5 

5 

M5 

5 

M5 

6 

M5 

6 

M5 

7 

M5 

7 

M5 

8 

M5 

8 

M5 

9 

M5 

9 

M5 

10 

M5 

10 

M5 

11 

M9 

17 

M5 

11 

M5 

12 

M9 

18 

M5 

12 

M5 

13 

M9 

19 

-- 

M5 

13 

M5 

lA 

M9 

20 

-- 

M5 

lA 

M5 

17 

M5 

17 

M7 

1 

M7 

1 

M7 

2 

M7 

2 

M7 

3 

M7 

3 

M7 

A 

M7 

A 

M7 

5 

M7 

5 

M7 

6 

M7 

6 

M7 

7 

M7 

7 

M7 

8 

M7 

8 

M7 

9 

M7 

9 

M7 

10 

M7 

10 

M7 

11 

M7 

11 

M7 

12 

M7 

12 

M7 

13 

M7 

13 

M7 

lA 

M7 

lA 

M7 

15 

M7 

15 

M7 

16 

M7 

16 

M8 

1 

M8 

1 

M8 

2 

M8 

2 

M8 

3 

M8 

3 

M8 

A 

M8 

A 

M8 

5 

M8 

5 

M8 

6 

M8 

6 

M8 

7 

MS 

7 

Table  A, 21 
(continued) 


FIELD 


COHORT  1 
BLOCK  ITEM 


COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


S211008 

-- 

M8 

8 

S211009 

-- 

M8 

9 

S211010 

-- 

-- 

M8 

10 

S211011 

-~ 

M8 

11 

S211012 

-- 

— 

M8 

12 

S211013 

-- 

-- 

M8 

13 

8211014 

-- 

— 

M8 

lA 

S211015 

-- 

-- 

M8 

15 

S211301 

-- 

-- 

M9 

1 

S211302 

-- 

M9 

2 

S211303 

-- 

M9 

3 

S21130A 

-- 

-- 

M9 

A 

S211305 

-- 

-- 

M9 

5 

S211306 

-- 

-- 

M9 

6 

S211307 

-- 

M9 

7 

S211308 

-- 

-- 

M9 

8 

S211/i01 

-- 

-- 

M9 

9 

S211402 

-- 

-- 

M9 

10 

S211A03 

-- 

-- 

M9 

11 

S211A0A 

-- 

-- 

M9 

12 

S211A05 

-- 

-- 

M9 

13 

S211A06 

-- 

M9 

lA 

S211A07 

-- 

M9 

15 

S211A08 

-- 

-- 

M9 

16 

S211A09 

-- 

-- 

-- 

-- 

S211A10 

-- 

-- 

-- 

-- 

S211501 

-- 

-- 

-- 

S211502 

-- 

-- 

-- 

-- 

S211503 

-- 

-- 

-- 

S21150A 

-- 

-- 

-- 

-- 

S211505 

-- 

-- 

-- 

S211506 

— 

-- 

-- 

S211507 

— 

-- 

-- 

-- 

S211508 

— 

-- 

-- 

-- 

S211509 

— 

-- 

-- 

-- 

S211510 

-- 

-- 

-- 

-- 

S211511 

-- 

-- 

-- 

S212001 

-- 

-- 

S212002 

S212003 

S21200A 

S212005 

S212006 

S212007 

8212008 

S212009 

S212010 

S212011 

S212101 

S212102 

S212103 

S21210A 

S212105 

M8 

8 

M8 

9 

M8 

10 

M8 

11 

M8 

12 

M8 

13 

M8 

lA 

M8 

-- 

15 


Ml 

3 

Ml 

A 

Ml 

5 

Ml 

6 

Ml 

7 

Ml 

8 

Ml 

9 

Ml 

10 

Ml 

1 

Ml 

2 

Ml 

1 

Ml 

2 

Ml 

3 

Ml 

A 

Ml 

5 

Ml 

6 

Ml 

7 

Ml 

8 

Ml 

9 

Ml 

10 

Ml 

11 

M9 

1 

M9 

2 

M9 

3 

M9 

A 

M9 

5 

M9 

6 

M9 

7 

M9 

8 

M9 

S 

M9 

10 

M9 

11 

M9 

12 

M9 

13 

M9 

lA 

M9 

15 

M9 

16 

-151- 


ERIC 


50,5 


Table  A. 22 
SCIENCE  BACKGROUND  ITEMS 


FIELD 

S400101 

SA00102 

S400103 

S400104 

S40010S 

S400106 

SA00107 

S400108 

SA00109 

SAOOllO 

SAOOlll 

SAOu^J'^l 

S400302 

S400303 

S400304 

SA00305 

SA00701 

SA00702 

SA00703 

S400801 

S400802 

S400803 

S400804 

SA00805 

SA00806 

S400901 

S400902 

S401201 

S401202 

S401203 

S401204 

SA01205 

SA01206 

SA01207 

S401208 

S401209 

S401301 

S401302 

S401303 

S401304 

S401305 

S401401 

S401402 

S401403 

S401404 

S401501 

S401601 

S401602 

S401603 

S401604 

S401605 

S401606 

S401607 

S401608 

S401701 

S401801 

S401901 

S401902 

S401903 

S401904 

S401905 

S401906 

S401907 

S401908 

S401909 

S401910 

S401911 

S401912 

S402001 

S402002 

S402003 

S402004 

S402005 


COHORT  1 
BLOCK  ITEM 


COHORT  2 
BLOCK  ITEM 


COHORT  3 
BLOCK  ITEM 


FIELD 


COHORT  1 
BLOCK  ITEM 


S3 
S3 
S3 
S3 
S3 
S3 
S3 
S3 
S3 
S3 
S3 
SX 
SI 
SI 
SI 
SI 


1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
1 
2 
3 
4 
5 


S5 

1 

S4 

2 

S4 

3 

S4 

4 

S4 

5 

S4 

6 

S4 

7 

S4 

8 

S4 

9 

S5 

2 

__ 

__ 

S402101 

S4 

1 

S4 

-- 

S402201 

-- 

S7 

__ 

-- 

-- 

S402202 

-- 

S7 

-- 

-- 

S402203 

-- 

S7 

-- 

S402204 

-- 

S7 

__ 

-- 

-- 

S402301 

-- 

-- 

S6 

__ 

S402302 

-- 

S6 

__ 

-- 

S402303 

-- 

S6 

__ 



-- 

-- 

S402304 

-- 

S6 

-- 

S40230S 

-- 

-- 

S6 



S402306 

-- 

S6 





-- 

S402307 

-- 

S6 



S402308 

-- 

S6 



S402309 

-- 

S6 







S402401 

-- 

S7 



S402501 

-- 

-- 

S7 

S2 

1 

S2 

1 

S402502 

-- 

S7 

S2 

2 

S2 

2 

S402503 

-- 

S7 

S2 

3 

S2 

3 

S402504 

-- 

S7 

S2 

4 

S2 

4 

S402601 

S8 

S2 

5 

S2 

5 

S402701 

S8 

S2 

6 

S2 

6 

S402702 

-- 

-- 

S8 

S2 

7 

S2 

7 

S402703 

-- 

S8 

S2 

8 

S2 

8 

S402704 

-- 

-- 

S8 

S2 

9 

S2 

9 

S402705 

-- 

S8 

SI 

1 

SI 

1 

S402801 

-- 

-- 

S8 

SI 

2 

SI 

2 

S402802 

-- 

-- 

S8 

SI 

3 

SI 

3 

S402303 

S8 

SI 

4 

SI 

4 

S402804 

S8 

SI 

5 

SI 

5 

S402805 

-- 

-- 

S8 

SI 

6 

SI 

6 

S402806 

-- 

-- 

S8 

SI 

7 

SI 

7 

S402807 

-- 

-- 

S8 

SI 

8 

SI 

8 

S402808 

-- 

S8 

SI 

9 

SI 

9 

S402901 

-- 

-- 

SI 

10 

SI 

10 

S402902 

-- 

-- 

SI 

11 

SI 

11 

S402903 

-- 

S3 

1 

S3 

1 

S402904 

-- 

S3 

2 

S3 

2 

S40290S 

-- 

-- 

S3 

3 

S3 

3 

S402906 

-- 

S3 

4 

S3 

4 

S402907 

S3 

5 

S3 

5 

S402908 

S3 

6 

S3 

6 

S402909 

S3 

7 

S3 

7 

S403101 

S9 

S3 

8 

S3 

8 

S403102 

S9 

S3 

9 

S3 

9 

S403103 

S9 

-- 

S403104 

-- 

-- 

S9 

S4 

2 

S4 

3 

S403105 

S9 

S4 

3 

S4 

4 

S403106 

-- 

S9 

S4 

4 

S4 

5 

S403107 

S9 

S4 

5 

S4 

6 

S403108 

S9 

S4 

6 

S4 

7 

S403201 

S6 

1 

S4 

7 

S4 

8 

S403301 

-- 

S4 

8 

S4 

9 

S403302 

-- 

-- 

S4 

9 

S4 

10 

S403303 

-- 

-- 

S6 

1 

S6 

1 

S403304 

-- 

-- 

S5 

1 

S5 

1 

S403305 

S7 

1 

S403306 

-- 

-- 

S7 

2 

S403307 

S7 

3 

S403308 

S7 

4 

S403309 

S7 

5 

S403310 

S7 

6 

S403401 

S7 

7 

S403501 

S7 

8 

S403502 

S7 

9 

S403503 

S7 

10 

S403504 

S7 

11 

S403505 

S7 

12 

S403601 

S5 

3 

S9 

1 

SO 

1 

S403701 

S5 

4 

S9 

2 

S8 

2 

S403801 

S6 

2 

S9 

3 

S8 

3 

S403901 

S6 

3 

S9 

4 

S8 

4 

S404001 

S6 

4 

S9 

5 

S8 

5 

S404101 

S7 

1 

COHORT  2 
BLOCK  ITEM 


1 
2 
3 
4 
5 
2 
3 
4 
5 
6 
7 
8 
9 
10 
1 
6 
7 
8 
9 
1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 


COHORT  3 
BLOCK  ITEM 


6 
7 
8 
9 
10 
11 
12 
13 


S4 
SI 
SI 
SI 

SI 

S6 

S6 

S6 

S6 

S6 

S6 

S6 

S6 

86 

S4 

SI 

SI 

SI 

SI 

S4 

SI 

SI 

SI 

SI 

SI 

S9 

S9 

S9 

S9 

S9 

SO 

S9 

S9 
S9 
S9 
S9 
S9 
S9 
S9 
S9 
S9 
S9 
S8 
S8 
S8 
S8 
S8 
S8 
S8 
S8 

SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
SI 
S7 
S7 
S7 
S7 
S7 


1 

1 

2 

3 

4 

2 

3 

4 

5 

6 

7 

8 

9 
10 
11 
12 
13 
14 
15 

2 
5 
6 
7 
8 
9 
10 
11 
12 
13 

14 

15 

16 

17 
1 
2 
3 
4 
5 
6 
7 
8 
9 
6 
7 
8 
9 

10 

11 

12 

13 

1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
13 
14 
15 
16 
17 


-452- 


ERLC 
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TAbl«  A. 22 

(continued) 


COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

BLOCK 

ITEM 

BLOCK  ITEM 

BLOCK  ITEM 

S404102 

S7 

2 

S404103 

S7 

3 

S404104 

S7 

4 

S404105 

S7 

5 

S404201 

S7 

6 

S404301 

S7 

7 

-453- 

507 


Table  A. 23 
COMPUTER  COMPETENCE  BACKGROUND  ITEMS 


COHORT  1 

COHORT  2 

COHORT  3 

COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

X  1  L^n 

ITEM 

BLOCK 

ITEM 

FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEM 

S600101 

r*  1 

^  X 

X 

^  X 

\ 

CI 

1 

S60080A 



CI 

13 

CI 

13 

S600102 

2 

\^  X 

2 

CI 

2 

__ 

C5 

2 

C6 

13 

S600103 

•a 
J 

^  X 

•a 

\^  X 

3 

S600805 

__ 



CI 

lA 

CI 

lA 

S60010A 

^  X 

CI 

S600806 





CI 

15 

CI 

15 

S600105 

5 

^  X 

5 

CI 

5 





C5 

A 

S600106 

^  X 

g 

^  X 

5 

CI 

6 

S600807 

__ 

CI 

16 

CI 

16 

S600201 

^  X 

7 

^  X 

7 

CI 

7 



C5 

5 

C6 

lA 

C2 

10 

C2 

10 

C2 

10 

S600808 

__ 

CI 

17 

CI 

17 

S600301 

\^  X 

Q 

CI 

3 

CI 

8 

S600809 





CI 

18 

CI 

18 

1  1 

C2 

X  X 

C2 

1 1 

S600810 





CI 

19 

CI 

19 

S600302 

CI 

O 
O 

^^x 

C 1 

8 



__ 

C5 

6 

C2 

X  X 

C2 

1 1 

C2 

11 

S600811 





C5 

3 

S600303 

^  X 

8 

CI 

8 

CI 

8 

S600812 

C6 

15 

C2 

X  X 

C2 

1 1 

C2 

1 1 

S600813 





C6 

16 

S60030A 

^  X 

Q 

^  X 

8 

CI 

8 

S600901 

C2 

1 

C2 

1 

C2 

1 

C2 

"J  "J 
X  X 

C2 

1 1 

C2 

11 

S600902 

C2 

2 

C2 

2 

C2 

2 

S600401 

\^  X 

g 

^  X 

g 

CI 

g 

S600903 

C2 

3 

C2 

3 

C2 

3 

S600501 

C 1 

X  u 

S60090A 

C2 

A 

C2 

A 

C2 

A 

S600  502 

CI 

1 1 

S600905 

C2 

5 

C2 

5 

C2 

5 

S600  503 

C 1 

S600906 

C2 

6 

C2 

6 

C2 

6 

S600  50A 

CI 

13 

S600907 

C2 

7 

C2 

7 

C2 

7 

S600  SOS 

%^  Sj  \J  \J    mJ  \J  mJ 

CI 

1 A 

S600908 

C2 

8 

C2 

8 

C2 

8 

S600  S06 

CI 

15 

S600909 

C2 

g 

C2 

9 

C2 

9 

S600507 

C 1 

XD 

S601001 

C2 

12 

C2 

12 

C2 

12 

S600508 

^x 

1  7 
X  ' 

S60X101 

C2 

13 

C2 

13 

C2 

13 

S600509 

^  X 

X  o 

C3 

9 

S600601 

CI 

0  u 

C 1 

0  X 

\*  X 

O  X 

S601201 

C3 

1 

CA 

1 

CO 

C2 

25 

C2 

25 

S601301 

C3 

2 

CA 

2 

C3 

22 

C3 

1  ft 
X  o 

C3 

18 

S601A01 

C3 

3 



CA 

25 

CA 

3 1 

S601501 

C3 

A 



__ 

__ 

C5 

18 

C5 

33 

S601601 

C3 

5 

__ 

__ 

__ 

-- 

C6 

21 

C6 

25 

S60I701 

C3 

6 

__ 

-- 

-- 

S600602 

CI 

30 

CI 

31 

CI 

31 

S601801 

C3 

7 

-- 

C2 

2A 

C2 

25 

C2 

25 

S601901 

C3 

8 

C3 

22 

C3 

18 

C3 

18 

S602001 

C3 

10 

C6 

10 

CA 

25 

CA 

31 

S602101 

C3 

11 

C3 

17 

C3 

17 

C5 

18 

r5 

33 

C5 

16 

C6 

21 

C6 

26 

S602201 

__ 

__ 

C3 

16 

C3 

16 

S600603 

\*  X 

30 

CI 

3 1 

CI 

31 

S602301 





CA 

3 

__ 

C2 

2A 

C2 

25 

C2 

25 

S602A01 



CA 

A 

-- 

03 

22 

C3 

18 

C3 

18 

S602501 





CA 

5 

CA 

21 

CA 

25 

CA 

31 

S602601 





C6 

1 

CA 

1 

C5 

18 

C5 

33 

S602701 



C6 

2 

CA 

2 

C6 

21 

C6 

26 

S602801 





C6 

3 

CA 

3 

S600604 

^  X 

30 

CI 

3 1 

CI 

31 

S602901 

C6 

5 

CA 

A 

C2 

2A 

C2 

25 

C2 

25 

S603001 





C6 

6 

CA 

5 

\*  0 

C3 

1  ft 
X  o 

C3 

18 

S603l0i 

__ 

C6 

7 

CA 

6 

CA 

25 

CA 

31 

S603201 



CA 

7 

C5 

18 

C5 

33 

S603202 

_  _ 

__ 

CA 

8 

C6 

21 

C6 

26 

S603203 



_- 

CA 

9 

S600605 

CI 

30 

CI 

31 

CI 

31 

S60320A 





CA 

10 

C2 

2A 

C2 

25 

C2 

25 

S603205 

__ 

-- 

-- 

CA 

11 

C3 

22 

C3 

18 

C3 

18 

S603206 

__ 

-- 

-- 

-- 

CA 

12 

CA 

25 

CA 

31 

S603207 

_- 

-- 

-- 

CA 

13 





C5 

18 

C5 

33 

S603208 

__ 

-- 

-- 

CA 

lA 

__ 



C6 

21 

C6 

26 

S603209 

-- 

-- 

-- 

CA 

15 

S600606 

CI 

30 

CI 

31 

CI 

31 

S603210 

-- 

-- 

-- 

CA 

16 

C2 

2A 

C2 

25 

C2 

25 

S603211 

-- 

C4 

17 

C3 

22 

C3 

18 

C3 

18 

S603212 

CA 

18 

CA 

25 

CA 

31 

S603213 

CA 

19 

C5 

18 

C5 

33 

S603301 

CA 

20 

C6 

21 

C6 

26 

S603A01 

C5 

10 

S600701 

CI 

31 

CI 

32 

CI 

32 

S603A02 

C5 

10 

C2 

25 

C2 

26 

C2 

26 

S603A03 

C5 

10 

C3 

23 

C3 

19 

C3 

19 

S603A0A 

C5 

10 

CA 

26 

CA 

32 

S603501 

C5 

11 

C5 

19 

C5 

3A 

S603502 

C5 

11 

C6 

22 

C6 

27 

S603503 

C5 

11 

S600801 

CI 

10 

CI 

10 

S60350A 

C5 

11 

C5 

1 

C6 

12 

S603601 

C5 

1 

S600802 

CI 

11 

CI 

11 

C6 

1 

S600803 

CI 

12 

CI 

12 

S603602 

C5 

2 

C6 
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Table  A. 23 
(continued) 


COHORT  1 
FIELD  BLOCK  ITEM 

S603603 

S60360A 

S603701 

S603702 

S603703 

S60370A 

S603801 
S603802 
S603803 
S60380A 
S603805 
S603806 
S603807 
S603901 
S60A101 
S60A201 
S60A301 
S60AA01 
S60AA02 
S60AA03 
S60A501 
S60A502 


COHORT  2  COHORT  3 

BLOCK    ITEM          BLOCK  ITEM 

C5  3 

C6  3 

C5  A 

C6  A 

C5  5 

C6  5 

C5  6 

C6  6 

C5  7 

C6  7 

C5  8 

C6  8 

C5  9 

C5  10 

C5  11 

C5  12 

C5  13 

C5  lA 

C5  15 

C5  17 

C6  A 
C6  8 
C6  9 

C6  9 

C6  10 

C6  11 

C6  17 

06  17 
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Table  A. 2^ 
U.S.  HISTORY  BACKGROUND  ITEMS 


COHORT  2 

COHORT 

3 

■RT  OPIC  ITEM 

BLOCK  ITEM 

BLOCK  ITEM 

H800101 

U  U  w  V  JL  w  JL 





HI 

1 



H2 

1 

H3 

1 

H4 

1 

RAnn^n 1 

HI 

2 

H2 

2 

H3 

2 

HA 

2 

HI 

3 

H2 

3 

H3 

3 

H4 

3 

nouu^UJ 

HI 

A 

__ 

H2 

A 



H3 

A 



HA 

A 

H800?04 

LiU  W  ^  V  ^ 





HI 

5 



H2 

5 

__ 



H3 

5 



__ 

HA 

5 

UO  V  V      V  .J 



HI 

6 



H2 

6 

__ 

HA 

6 

0000301 



HI 

7 

__ 

H2 

7 

H3 

7 

HA 

7 

no  uu  o  u  £i 

HI 

8 

H2 

8 

H3 

8 

HA 

8 

H800303 



HI 

9 



H2 

9 



H3 

9 



HA 

9 

H800304 

HI 

10 





H2 

10 



H3 

10 



HA 

10 

H800305 



__ 

HI 

11 



H2 

11 



H3 

11 



HA 

11 

H800306 





HI 

12 

__ 



H2 

12 



H3 

12 

__ 

__ 

HA 

12 

H800401 





HI 

A9 



H2 

A9 



H3 

AC 



HA 

A7 

H800402 





HI 

50 

^  «  «. 



H2 

50 





H3 

A9 



^_ 

HA 

A8 

H800403 

HI 

51 



H2 

51 





H3 

50 





HA 

A9 

H800A04 

HI 

52 

H2 

52 

H3 

51 

HA 

50 

H800A05 

HI 

53 

H2 

53 

H3 

52 

HA 

51 

H800A06 

HI 

5A 

H2 

5A 

H3 

53 

HA 

52 

ERIC 


COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

BLOCK  ITEM 

BLOCK  ITEM 

BLOCK 

ITEM 

H800501 

HI 

55 





H2 

55 



H3 

5A 



HA 

53 

H800502 





HI 

56 





H2 

56 





H3 

55 





HA 

5A 

H800503 



__ 

HI 

57 



H2 

57 



H3 

56 

w_ 

__ 

HA 

55 

H80050A 

__ 

HI 

58 

__ 

H2 

58 

H3 

57 

HA 

56 

H800505 

HI 

59 

H2 

59 

H3 

58 

HA 

57 

H800506 

HI 

60 

H2 

60 

H3 

59 

HA 

58 

H800507 

HI 

61 

H2 

61 

H3 

60 

HA 

59 
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Table  A. 25 
LITERATURE  BACKGROUND  ITEMS 


COHORT  1  COHORT  2  COHORT  3 


FIELD 

BLOCK 

ITEM 

BLOCK 

ITEM 

BLOCK 

ITEh 

L800101 

LI 

1 

-- 

L2 

1 

— 

— 

L3 

1 

-- 

-- 

L4 

1 

L800201 

-- 

LI 

2 

L2 

2 

L3 

2 

-- 

-- 

L4 

2 

L800301 

-- 

-- 

LI 

3 

-- 

L2 

3 

-- 

L3 

3 

-- 

-- 

L4 

3 

L800401 

-- 

-- 

LI 

A 

-- 

-~ 

L2 

A 

-- 

-- 

L3 

A 

-- 

-- 

L4 

A 

L800501 

-- 

LI 

5 

-- 

L2 

5 

-- 

L3 

5 

-- 

-- 

L4 

5 

L800601 

-- 

-- 

LI 

5 

-- 

-- 

-- 

L2 

5 

-- 

-- 

L3 

5 

-- 

LA 

5 

L800602 

-- 

~- 

LI 

6 

-- 

__ 

-- 

-- 

L2 

6 

-- 

-- 

-- 

L3 

6 

-- 

-- 

LA 

6 

L800603 

— 

LI 

7 

— 

— 

— 

L2 

7 

— 

— 

— 

L3 

7 

-- 

-- 

LA 

7 

L800604 

— 

— 

— 

LI 

8 

— 

-- 

— 

L2 

8 

— 

— 

L3 

8 

— 

— 

LA 

8 

L800605 

— 

— 

— 

— 

LI 

9 

— 

— 

__ 

-- 

L2 

9 

— 

-- 

-- 

L3 

9 

-- 

-- 

-- 

LA 

9 

L800606 

— 

— 

-- 

-- 

LI 

10 

— 

-- 

-- 

L2 

10 

-- 

L3 

10 

-- 

LA 

10 

L800607 

— 

-- 

LI 

11 

— 

— 

-- 

L2 

11 

— 

L3 

11 

-- 

LA 

11 

L800701 

— 

LI 

12 

— 

-- 

__ 

L2 

12 

L3 

12 

LA 

12 

L800702 

— 

— 

-- 

LI 

13 

— 

— 

~- 

-- 

L2 

13 

— 

-- 

-- 

L3 

13 

LA 

13 

L800703 

— 

-- 

LI 

lA 

— 

L2 

lA 

— 

— 

L3 

lA 

LA 

lA 

L800704 

LI 

15 

L2 

15 

L3 

15 

LA 

15 

L800705 

LI 

16 

L2 

16 

L3 

16 

LA 

16 

L800801 

LI 

17 

L2 

37 

L3 

17 

LA 

17 
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COHORT  1 

COHORT  2 

COHORT  3 

FIELD 

BLOCK  ITEM 

BLOCK  ITEM 

TTFM 

L800901 





LI 

18 





L2 

18 



T 

1  ft 





T  L 

1  ft 

L801001 



LI 

A  9 



L2 





L  J 

A9 



T  L 

A  9 

L801101 

LI 

D  U 



L2 

CI 

D  X 





T 

C  rt 
DU 





T  L 

Li  h 

C  f\ 
DU 

L801201 



Ll 

51 





L2 

52 

__ 



T 

J 1 



T  L 

D  X 

L801202 



Ll 

52 

__ 



L2 

53 

__ 



T  1 

52 





T  L 

L801203 



Ll 

53 





L2 

5  A 



T  T 
Ll  J 

DO 





LA 

J  J 

L80120A 





Ll 

5A 



L2 

55 



L3 

5  A 

__ 



LA 

5A 

L801205 



Ll 

55 

L2 

56 

__ 

__ 

L3 

55 

__ 

LA 

55 

L801206 

Ll 

56 

-- 

-_ 

L2 

57 

__ 

__ 

L3 

56 

__ 

LA 

56 

L801207 

__ 

Ll 

57 

__ 

-_ 

L2 

58 

__ 

__ 

L3 

57 

__ 

__ 

LA 

57 

L801208 

__ 



Ll 

58 

__ 

__ 

L2 

59 

__ 

L3 

58 

__ 

__ 

LA 

58 

L801209 

Ll 

59 

L2 

60 

__ 

-- 

L3 

59 

__ 

LA 

59 

L801301 

__ 

Ll 

60 

-- 

__ 

L2 

61 

__ 

L3 

60 

__ 

LA 

60 

L801302 

-_ 



Ll 

61 

__ 

L2 

62 

__ 

L3 

61 

LA 

61 

L801303 

-_ 

__ 

Ll 

62 

-- 

L2 

63 

L3 

62 

LA 

62 

L801A01 

Ll 

63 

L2 

6A 

L3 

63 

LA 

63 

L801A02 

Ll 

6A 

L2 

65 

L3 

6A 

LA 

6A 

L801A03 

Ll 

65 

L2 

66 

L3 

65 

LA 

65 

ERIC 
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Table  A, 25 
(continued) 


rOHfjRT  1 

COHORT  2 

COHORT  3 

PTFT  n 

OLl\J\ftK            Jk  A  till 

BLOCK  ITEM 

BT  OCK 

ITEM 

IrfO  V/  X  *t  *t 



LI 

66 

L2 

67 

L3 

66 

LA 

66 

Li  O  U    J.  ^    W  mJ 

LI 

67 

__ 

L2 

68 

L3 

67 

LA 

67 

T  801406 

... 

LI 

68 

L2 

69 

L3 

68 

LA 

68 

L801A07 

LI 

69 

L2 

7  0 

L3 

69 

LA 

69 

L801408 

LI 

70 

L2 

71 

L3 

70 

LA 

70 

L801A09 

LI 

71 

L2 

72 

L3 

71 

LA 

71 

L801A10 

LI 

72 

L2 

73 

L3 

72 

LA 

72 
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APPENDIX  B 
Conditioning  Variables  Tables 


Appendix  B 
CONDITIONING  VARIABLES  TABLES 


Appendix  B  contains  tables  of  estimated  effects  for  the  conditioning 
variables  used  in  the  construction  of  plausible  values  for  reading, 
mathematics,  and  science. 

Tables  B.l  through  B.3  show  the  effects,  by  grade/age,  for  reading. 

Tables  B.4,  B.6,  and  B.8  give  the  conditioning  variables,  by  grade/age, 
used  for  all  mathematics  subscales.     The  estimated  effects  for  these 
conditioning  variables  appear  in  Tables  B.5,  B.7,  and  B.9.     Tables  B.IO 
through  B.20  are  for  trend  in  mathematics.     Each  table  gives  conditioning 
variables  and  their  estimated  effects  for  a  given  age  and  assessment  year. 

Similarly,   the  conditioning  variables  for  the  science  subscales  appear 
in  Tables  B.21,  B.23,  and  B.25  and  the  estimated  effects  in  Tables  B.22, 
B.24,  and  B.26.     Tables  B.27  through  B.37  give  the  conditicuing  variables  and 
effects,  by  age  and  assessment  year,  for  science  trend. 

Note  that  all  effect  e  ,timates  are  in  the  metrics  used  in  the  original 
calibration  of  the  scale  or  subscale.     The  transformations  needed  to 
represent  ttiese  effects  in  terms  of  the  metric  of  the  final  reporting  scales 
appear  in  the  chapters  which  describe  the  scaling  of  each  subject  area.  Note 
also  that  certain  conditioning  variables  do  not  have  effect  estimates.  This 
is  because  those  variables  are  exact  linear  combinations  of  the  other 
conditioning  variables. 
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Table  B.l 

Estimated  Effects  for  Reading  Conditioning  Variables,  Grade  3/Age  9 


VARIABLE  ESTIMATED 

LABEL  EFFECT  DESCRIPTION 

1  OVERALL  -1.416868     1  OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2  GENDER2  0.134737     2  SEX  (FEMALE) 

3  ETHNIC2  -0.151446     3  ETHNICITY  (BLACK) 

4  ETHNIC3  -0.232955    4  ETHNICITY  (HISPANIC) 

5  ETHNIC4  -1.638121     5  ETHNICITY  (ASIAN  AMERICAN) 

6  ST0C2  0.157261     6  SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7  ST0C3  0.131141     7  SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8  REGI0N2  -0.066794    8  REGION  (SOUTHEAST) 

9  REGI0N3  -0.007434    9  REGION  (CENTRAL) 

10  REGI0N4  -0.066843  10  REGION  (WEST) 

11  PARED2  0.088011  11  PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12  PARED3  0.211662  12  PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13  PARED4  0.242878  13  PARENTS  EDUCATION  (COLLEGE  GRAD) 

14  PARED_  0.139631  14  PARENTS  EDUCATION  (MISSING.   I  DON'T  KNOW) 

15  ITEMS2  0.071657  15  ITEMS  IK  HOME  (FOUR  OF  THE  FIVE) 

16  ITEMS3  0.123210  16  ITEMS  IN  HOME  (FIVE  OF  THE  FIVE) 

17  TV  0.103606  17  HOURS  TV  WATCHING  (LINEAR) 

18  TV**2  -0.019185  18  HOURS  TV  WATCHING  (QUADRATIC) 

19  HW-YES  0.043739  19  HOMEWORK  (YES  -  SOME  AMOUNT) 

20  HW-2345  -0.024127  20  HOMEWORK  AMOUNT  (LINEAR) 

21  LM  BY  E3  0.028126  21  LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISPANIC) 

22  LM  BY  E4  0.334282  22  LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  ASIAN  AMER) 

23  LM  BY  E_  -0.067490  23  LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  OTHER  ETH) 

24  LUNCH%  -0.080313  24  PERCENT  IN  LUNCH  PROGRAM 

25  LUNCH_  -0.082898  25  LUNCH  PROGRAM  (MISSING) 

26  %WHITE49  -0.142514  26  PERCENT  WHITE  IN  SCHOOL  (0-49%  WHITE  MINORITY) 

27  %WHITE79  -0.041980  27  PERCENT  WHITE  IN  SCHOOL  (50-79%  INTEGRATED) 
%WHITEOO  28  PERCENT  WHITE  IN  SCHOOL  (80-100%  PREDOMINANTLY) 

28  E2  X  SEX  0.127858  29  ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

29  E3  X  SEX  0.098589  30  ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

30  E4  X  SEX  0.040797  31  ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

31  E2  X  PE2  -0.142674  32  ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

32  E2  X  PE3  -0.224876  33  ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

33  E2  X  PE4  -0.066695  34  ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

34  E2  X  PE_  -0.054574  35  ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

35  E3  X  PE2  -0.044825  36  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

36  E3  X  PE3  -0.180012  37  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

37  E3  X  PE4  -0.147884  38  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

38  E3  X  PE_  0.014333  39  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

X  PE2  40  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

39  E4  X  PE3  1.115009  41  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,   POST  HS) 

40  E4  X  PE4  1.483300  42  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

41  E4  X  PE_  1.427857  43  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

42  MA,<MG  -0.399390  44  MODAL  AGE,  LESS  THAN  MODAL  GRADE 
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Table  B.l 
(continued) 


TTAD  T  ART  V 

ESTIMATED 

EFFECT 

DESCRIPTION 

4j 

0. 

029070 

45 

MnDAT  AGE    MODAL  GRADE  MISSING 

A  A 

0. 

414784 

46 

MnnAT   AGF     GREATER  THAN  MODAL  GRADE 

N^iur  A  Mr" 

-0. 

263028 

47 

r^RFATFR  THAN  MODAL  AGE    MODAL  GRADE 

46 

C/^TJ  T'VDt? 

bOn  ilrrj 

0. 

026332 

48 

^rHOOT   TYPF   ^'NOT  PUBLICS 

4/ 

ACV  CTJ9 

0. 

050731 

49 

FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

Aft 

1:  IxCjD  v^ni. 

0. 

077444 

50 

WENT  TO  PRESCHOOL  (YES) 

:/iPARFNTl 

0. 

175494 

51 

SINGLE/MULTIPLE  PARENT  HOME  (MOTHER, FATHER  HOME) 

50 

MOTHER 

-0. 

027370 

52 

MOTHER  AT  HOME 

51 

MOWORK 

0 

012694 

53 

MOTHER  WORKS  OUTSIDE  (YES) 

52 

SCI123 

0 

.054960 

54 

SPENT  AT  LEAST  ONCE  A  WEEK  STUDYING  SCIENCE 

53 

SCI45 

0 

.019661 

55 

SPENT  <  ONCE  A  WEEK  OR  NEVER  STUDYING  SCIENCE 

54 

COMPUTER 

0 

.022764 

56 

USE  COMPUTERS  FOR  MATH,  READING,  ETC.  (YES) 

55 

SUPERVIS 

0 

.071617 

57 

ADULT  SUPERVISION  OF  STUDENT  AFTER  SCHOOL(YES) 

56 

MATH  Ql 

-0 

.274396 

58 

MATH    QUANTILE  (LINEAR  -1,0,1) 

57 

SCI  Ql 

-0 

.281302 

59 

SCIENCE    QUANTILE  (LINEAR  -1,0,1) 

58 

SAMPLE 

-0 

.445823 

60 

BIB  SAMPLE 
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Table  B.2 


Estimated  Effects  for  Reading  Conditioning  Variables,  Grade  7/Age  13 


VARIABLE 

ESTIMATED 

LABEL 

EFFECT 

UrjoUKXr  i  iUW 

1 

X 

n\7T?D  A  T  T 

-0. 

943157 

1 

X 

OVERALL  CONSTANT  ' 1 '   FOR  EVERYONE 

9 

0. 

160314 

9 
Z 

C  l?V     /  T7T7Iur  A  T  T7  \ 

b  riA  (  r  riMALli ) 

'X 

-0 

075165 

o 

liinNiUii  X  (BLACK) 

A 
*+ 

XL  IXlTi  XL>  J 

-0 

071093 

A 

hinWiuiiY  (HISPANIC) 

s 

-0 

077456 

h  i  nW  i  u  i  i I   ( Ab IAN  AMERI CAN ) 

6 

ST0C2 

0. 

091970 

C. 
u 

biz.ri  AWU  iirii  Or  CUMMUNixi    (HIGH  METRO) 

7 

ST0C3 

0 

050839 

7 

oiz.li  AJNU  iirn  Ur  uUnriUJNiii   (NOT  HIGH  OR  LOW) 

8 

RFGTON!? 

-0 

016276 

g 

ixlLVjlUri     oUU inJiAb  i  ) 

9 

REGI0N3 

-0 

039972 

Q 

-7 

10 

RFGT0N4 

-0 

062560 

10 

XUIN     ^  WIL^  1  } 

11 

PARED 2 

-0 

027861 

11 

12 

PARED 3 

0 

086129 

12 

13 

PARED4 

G 

032187 

13 

PARFNTS   PTiTTrATTOM      TOT  T  RPR  PPAn\ 

14 

PARED 

-0 

067058 

14 

15 

ITEMs7 

0 

086084 

1  S 

TTRM^    TM  MnMR    /"PnTTP    m?  TUT?  TTT7T7\ 
X  JL i:«no   XLN  nufj.li   ^ r uurv.  ur   inH  r  x VH^ 

16 

0 

121662 

1 

iiJii'lb  iW  hUnii  (riVil  OF  IHE  FIVE) 

17 

TV 

0 

092448 

17 

VinTTPQ    T\7    TJATr'lJTKI/^  /TTMITADN 

nuUKo   i  V  WAiunxDib   ( JLxMnAK) 

18 

XV**  2 

-0 

016185 

18 

19 

HW-NO 

-0 

222397 

19 

20 

HW-YES 

-0 

150057 

20 

HOMFUORK   /'YFS    -    SOMF  AM^TTWT^ 

21 

HW-2345 

0 

018195 

21 

HOMFUOR?^   AMOTTMT  /'TT^JRAP^ 
nvJXT-IjWUIvrs.  ATIUUIN  i    ^  LiXlNCiiAJCv. ^ 

22 

LM  BY  E3 

0 

042232 

22 

T  ANJriTTAf^F   MTKff"iPTTV   RV   "CTUMT r'T TV    /"vtrc  UTCPAMTr«\ 
XiMiN^uri^iL  n.xiNUi\.x i  I   13 X  HinJNxUlix    (xiLo,    n I o r AJN I L» ) 

23 

LM  BY  E4 

-0 

038987 

23 

T  AMfiTTAriF   MTMOPTTV   RV   UTUMTr'TTV    /vtrc       act  am  awtttdn 
XifUN^u/i^iL  nxiNUKXii  Dx  IL inlN xL» X i  I    (xriO,   AblAN  AMllK) 

24 

LM  BY  E 

-0 

067731 

24 

T  AMPTTAPR   MrKfnPTTV   RV  PTUMTr"TTV    /"vtrc      i^'T«ur?n  tttun 

XirtJNkjUAkjiL  nxiNUKXXi  Ux  HinTJiui i  X   (xriO,   OIHER  liiH) 

25 

LUNCHX  " 

0 

049258 

25 

PFRflFNT  TN  T  TTMrVI  PPOPPAM 

26 

LUNCH 

-0 

063302 

26 

T  TTNrH  PROrjRAM  /'MTSSTMP^ 

27 

%WHITE49 

-0 

074382 

27 

PFRflFNT  UHTTF   TNf  srVTOOT    ^H-AQy  tjuttt?  UTKinPTTVN 
X             X   wnxxij   xix   OL»nu\JXj   ^u-M-^A   WnXiH  nXWUKXxx^ 

28 

XWHITE79 

0 

023343 

28 

PFRPFMT  UHTTR    TM   QPUnnT     /'^H   "7Qj    txttttod  a tttf* \ 
IT iLis-UiLiN X  wnxiiL  xiN  oUnuULi  ^jU-/y/t   IN iilCjKAirjD) 

29 

XWHITEOO 

29 

PFRflFNT  UHTTF   TKf  <3rVIOOT    ( 1 0ny   PPTrnnMTMAMTT  v\ 

E2  X  SEX 

0 

001750 

30 

FTHNTflTTY  RY  PRMfiRP    ^RT  APV  ^7RMATR^ 
iLxxixixoxxx  ij X  vjCjIX jjCjIx  i^ijx^ujs,  r r«nALirj ^ 

30 

E3  X  SEX 

0 

029345 

31 

ETHNICITY  BY  GENDER  ^'HISPANIC  FFMATF^ 

31 

E4  X  SEX 

-0 

008379 

32 

FTHNTflTTY  RY  PFNTiFR    ^A^JTAM  AMRPTPAKT  t?17MatT7\ 

32 

E2  X  PE2 

-0 

052449 

33 

FTHNTTTTY  RY  PARRMT'<5  Rn   /RTAPV     uc  r'DAn^ 
iLxruixuxxx   13 X   xrixvCjiX X   o   cu   ^^IjXAL»J\.,    no  vjKAlJ^ 

33 

E2  X  PE3 

-0 

084803 

34 

FTHNTflTTY  RY  PAPRMT'd   RFi    /RTAPV     pncT  uc\ 

34 

E2  X  ?E4 

-0 

120141 

35 

ETHNICITY  BY  PARENT'S  ED  ^BLACK    TOT LFGF  GRAD^ 

35 

E2  X  PE_ 

-0 

036973 

36 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

36 

E3  X  PE2 

-0 

170683 

37 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

37 

E3  X  PE3 

-0 

227141 

38 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,   POST  HS) 

38 

E3  X  PE4 

-0 

175748 

39 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

39 

E3  X  PE_ 

-0 

127979 

40 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

40 

E4  X  PE2 

0 

012545 

41 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

41 

E4  X  PE3 

-0 

141419 

kl 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

42 

^    E4  X  PE4 

0 

220770 

43 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,   COLL  GRAD) 

43 

E4  X  PE_ 

0 

103496 

44 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 
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Table  B.2 
(continued) 


VARIABLE  ESTIMATED 

LABEL  EFFECT  DESCRIPTION 

44  MA,<MG  -0.362952  45  MODAL  AGE,  LESS  THAN  MODAL  GRADE 

45  MA,MG  -0.046980  46  MODAL  AGE,  MODAL  GRADE,  MISSING 

46  MA,>MG  0.129974  47  MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

47  >MA,MG  -0.284088  48  GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

48  SCH  TYPE  0  153836  49  SCHOOL  TYPE  (NOT  PUBLIC) 

49  ASK  SW?  0.022892  50  FAMILY  ASKS  ABOUT  SCHOOLWORK  (AUMIOST  EVERY  DAY) 

50  PRESCHl  0.060263  51  WENT  TO  PRESCHOOL  (YES) 

51  #PARENT1  -0.012825  52  SINGLE/MULTIPLE  PARENT  HOME  (MOTHER, FATHER  HOME) 

52  MOTHER  -0.025736  53  MOTHER  AT  HOME 

53  MOWORK  -0.015841  54  MOTHER  WORKS  OUTSIDE  (YES) 

54  COMPUTER  -0.068049  55  USE  COMPUTERS  FOR  MATH,  READING,  ETC.  (YES) 

55  MATH2  0.164571  56  TYPE  OF  MATH  CL^SS (REGULAR  MATH) 

56  MATH3  0.241884  57  TYPE  OF  MATH  CL^SS (PRE -ALGEBRA) 

57  MATH45  0.1893V7  58  TYPE  OF  MATH  CL^SS (ALGEBRA) 

58  SCIENCE2        0.151230  59  STUDYING  IN  SCIENCE  THIS  YEAR  (LIFE  SCIENCE) 

59  SCIENCE3        0.108702  60  STUDYING  IN  SCIENCE  THIS  YEAR  (PHYSICAL  SCIENCE) 

60  SCIENCE4        0.112368  61  STUDYING  IN  SCIENCE  THIS  YEAR  (EARTH  SCIENCE) 

61  SCIENCE5        0.195202  62  STUDYING  IN  SCIENCE  THIS  YEAR  (GENERAL  SCIENCE) 

62  GRADES  0.236329  63  GRADES  IN  SCHOOL  (LINEAR) 

63  MATH  Ql  -0.150790  64  MATH    QUANTILE  (LINEAR  -1,0,1) 

64  SCI    Ql  -0.260019  65  SCIENCE    QUANTILE  (LINEAR  -1,0,1) 

65  SAMPLE  -0.110011  66  BIB  SAMPLE 
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Table  B.3 


Estimated  Effects  for  Reading  Conditioning  Variables,  Grade  11/Age  17 


VAKIAJBLE 

ESTIMATED 

LABEL 

EFFECT 

DESCRIPTION 

1 

/>T7T7T>  ATT 

OVERALL 

-0. 

459690 

I 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

o 

z 

GENDERz 

0. 

208740 

z 

SEX  (FEMALE) 

1 

ETHNICz 

-0. 

179031 

o 

ETHNICITY  (BLACK) 

A 

^^ 

-0. 

190122 

A 

4 

ETHNICITY  (HISPANIC) 

c 

J 

-0. 

375032 

C 

D 

ETHNICITY  (ASIAN  AMERICAN) 

c 
0 

STOCz 

0. 

155906 

c 
D 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0. 

064686 

-7 
/ 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

0 

o 

REGiONz 

-0. 

004313 

0 

o 

REGION  (SOUTHEAST) 

Q 

y 

REG  I  ON  3 

0. 

037366 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

-0. 

024202 

10 

REGION  (WEST) 

r ARE Dz 

-0. 

015655 

ii 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

1  0 

T5  AD  T7'r\0 

rARED^ 

0. 

060401 

1  o 
J.Z 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

1  7 

D  AD  TT'n/. 

0. 

089310 

J.  J 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

1  /. 
i^f 

DAD  TT'n 

-0. 

215376 

14 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

1  c 
i  J 

i lEMbz 

0 

108733 

1  c 

13 

ITEMS  IN  HOME  (FOUR  OF  THE  FIVE) 

0, 

141402 

ID 

ITEMS  IN  HOME  (FIVE  OF  THE  FIVE) 

1  7 

i  V 

0, 

028707 

1  7 

U/MTD  O     TTT    TT  A  T^OUT  XT/**      /T  TXTT^AT^N 

HOURb  TV  WATCHING  (LINEAR) 

1  V  ^^Z 

-0 

008759 

1  ft 

i-O 

nUUKo   Iv   WAlGnlING   (^QUADKAl  IG) 

19 

J.  7 

-0 

405091 

i  7 

T4nM'"R'T.Tnp V  /'nr^M'T  uaxtt?  axtv\ 

20 

-0 

175868 

90 
zu 

nUrllLWUXxJS.       lEiO    -    oUflEi   ArlUUIN  1  y 

91 

nw - z  J 

-0 

003929 

Z  1 

lJr\M'T7TTr\D  V     A\1/^TTXTP  /TTXTCADN 

nUflEiWUKK  AMOUNi  (LINEAR) 

99 

-0 

.075719 

9  9 
Z  Z 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISPANIC) 

T  M    RV  T7/i 

-0 

.170266 

9*^ 

LANGUAGE  MlNORlli  BY  ETHNICITY  (YES,   ASIAN  AMER) 

0 

.016883 

z*+ 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,   OTHER  ETH) 

9S 

-0 

.122057 

9  S 

DCD/^CXTT"     TXT    T  TTKT/^U  DD/^/^DAVJ 

rERGENl   IN  LUNCH  PROGRAM 

96 

-0 

.037937 

9  A 
zo 

T  TTMr'lJ    DDnr'DAK/f     /MT  CO  TKir*  \ 

97 

0 

.012205 

97 

FERCENi  WHITE  IN  SCHOOL  (0-49X  WHITE  MINORITY) 

98 

0 

.036887 

9ft 
z  o 

rc.KGc.INl  Wnllc.  IN  oGrlUUL  (3U-/yX  INIEGRAIED) 

9Q 

£.  7 

DTTD/^TrMT"    TTLITT'T?     TXT    0/^Uf\f\T      /OA     1  AAV     Dn  T^T\/>\_/x  XT  A  \TT«T  Xrx 

r'c.KGc.INl  WnllE  IN  bCHOOL  (oO-lOOX  PREDOMINANTLY) 

9Q 

F9  Y  9 FY 

-0 

.144274 

CinlN  1L»1 1 X   Di   L»c.INUc.K   v^BLAGK  rcJlALc.^ 

30 

E3  Y  SFY 

0 

.006474 

FTHMTr'TTV   RV   PT7MnT7P    /UTCPAMTr*  T?T?MATT7\ 

cin.ni.L»i.  1 1  DI  vjCINUCin.  v^nlor AJNIU  rjiinAL»c.y 

17/,    Y  cpY 

0 

.110695 

'^9 

lyTUMT/^T  T^V    DV    OT7XTTM7D      /  A  C  T  A  XT     AUrT7DT/^AXT     t:»TM#  A  T  T?  \ 

c.lrlINlGiii   DI  GC.INUC.K   v^AblAN  AtlEKlGAN  rEMALE) 

F9   Y  PF9 
HZ  A  ircz 

-0 

.063329 

c.lnINlGlli  DI  rARENl  b  ED  (BLACK,   HS  GRAD) 

F9   Y  PF'^ 
HZ    A  rCsJ 

0 

.047861 

34 

■CPUXTT /^TT'V     D\7     D  A  D  T7XTT' #  O     T7T\      /TtT  A                 D/^r»T«  ITr»\ 

ElHNlGlli  DI  PARENT  S  ED   (BLACK,   POST  HS) 

34 

F9  Y  PF^. 

-0 

.065619 

35 

FTHMTr'TTV    UV    PAD  T7MT  '  C     VTS     /OTA  r*V        r*r\J  J  T70T7    OD  A  r\  \ 

CiiliNiUlli   Dl   rAKCilNi   b   £jU    QdLAGK,    GuLLEGE  GRAU) 

35 

E2  X  PE_ 

0 

.103087 

36 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

36 

E3  X  PE2 

-0 

.041342 

37 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

37 

E3  X  PE3 

0 

.032451 

38 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,   POST  HS) 

38 

E3  X  PE4 

0 

.008064 

39 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

39 

E3  X  PE 

0 

.179269 

40 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

40 

E4  X  PE2 

0 

.081702 

41 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

41 

E4  X  PE3 

0 

.311960 

42 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

42 

E4  X  PE4 

0 

.247000 

43 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

43 

E4  X  PE_ 

0 

.388056 

44 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 
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Table  B.3 
(continued) 


VARIABLE 

ESTIMATED 

LABEL 

EFFECT 

DESCRIPTION 

44 

KA,<MG 

-0 
\j . 

45 

MODAL  AGE,  LESS  THAN  MODAL  GRADE 

45 

MA,MG 

-0 . 

053659 

46 

MODAL  AGE,  MODAL  GRADE,  MISSING 

46 

MA,>MG 

0 . 

027352 

47 

MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

47 

>MA,MG 

-0 . 

250503 

48 

GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

48 

SCH  TYPE 

0. 

063927 

49 

SCHOOL  TYPE  (NOT  PUBLIC) 

49 

ASK  SW? 

-0, 

034854 

50 

FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

50 

PRE S CHI 

0. 

001747 

51 

WENT  TO  PRESCHOOL  (YES) 

51 

#PARENT1 

0. 

008337 

\y  \j  \j     *y  1 

52 

SINGLE/MULTIPLE  PARENT  HOME  (MOTHER, FATHER  HOME) 

52 

MOTHER 

-0, 

041225 

53 

MOTHER  AT  HOME 

53 

MOWORK 

0 

000683 

\j  \j  \j  \j  \j  «^ 

54 

MOTHER  WORKS  OUTSIDE  (YES) 

54 

GRADES 

0 

215902 

55 

GRADES  IN  SCHOOL  (LINEAR) 

55 

PR0GRAM2 

0 

133872 

56 

HIGH  SCHOOL  PR0GRAM(C0LLEGE  PREPARATORY) 

56 

PR0GRAM3 

-0 

.046951 

57 

HIGH  SCHOOL  PROGRAM (VOCATIONAL) 

57 

MATH 

0 

.075715 

58 

NUMBER  OF  MATH  COURSES 

58 

sen  MCE 

0 

.067684 

59 

NUMBER  OF  SCIENCE  COURSES 

59 

F.')ST.EC2 

0 

.066029 

60 

TWO-YEAR  COLLEGE 

60 

POST  .EC3 

0 

.178022 

61 

FOUR-YEAR  COLLEGE 

61 

HRS  WORK 

-0 

.067968 

62 

HOURS  OF  OUTSIDE  WORK 

62 

ENGL23 

0 

.122914 

63 

TYPES  OF  ENGLISH  CLASS  (ADVANCED &C0LLEGE  PREP.) 

6i 

ENG^ISH5 

-0 

.193832 

64 

TYPES  OF  ENGLISH  CLASS (REMEDIAL) 

64 

MATH  Ql 

-0 

.175274 

65 

MATH    QUANTILE  (LINEAR  -1,0,1) 

SCI  Ql 

-0 

.288030 

66 

SCIENCE    QUANTILE  (LINEAR  -1,0,1) 

ERIC 
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Table  B.4 


Mathematics  Conditioning  Variables,  Grade  3/Age  9 


VARIABLE 

LABEL 

DESCRIPTION 

1 

OVERALL 

1 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

3 

ETHNICITY 

(BLACK) 

4 

ETHNIC3 

4 

ETHNICITY 

(HISPANIC) 

5 

ETHNIC4 

5 

ETHNICITY 

(ASIAN  AMERICAN) 

6 

ST0C2 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

8 

REGION  (SOUTHEAST) 

9 

REGI0N3 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

10 

REGION  (WEST) 

11 

PARED 2 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

ITEMS2 

15 

ITEMS  IN  HOME  (FOUR  OF  THE  FIVE) 

16 

ITEMS 3 

16 

ITEMS  IN  HOME  (FIVE  OF  THE  FIVE) 

17 

TV 

17 

HOURS  TV  WATCHING  (LINEAR) 

18 

TV**2 

18 

HOURS  TV  WATCHING  (QUADRATIC) 

19 

HW- 

-YES 

19 

HOMEWORK  (YES  -  SOME  AMOUNT) 

20 

HW- 

•2345 

20 

HOMEWORK  AMOUNT  (LINEAR) 

21 

LM 

BY  E3 

21 

LANGUAGE  MINORITY  Bv  ETHNICITY  (YES,  H  "PANIC) 

22 

LM 

BY  E4 

22 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  ASI^lN  AMER) 

23 

LM 

BY  E 

23 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  OTHER  ETH) 

24 

LUNCH% 

24 

PERCENT  IN  LUNCH  PROGRAM 

25 

LUNCH 

25 

LUNCH  PROGRAM  (MISSING) 

26 

XWHITE49 

26 

PERCENT  WHITE  IN  SCHOOL  (0-49%  WHITE  MINORITY) 

27 

%WHITE79 

27 

PERCENT  WHITE  IN  SCHOOL  (50-79%  INTEGRATED) 

XWHITEOO 

28 

PERCENT  WHITE  IN  SCHOOL  (80-100%  PREDOMINANTLY) 

28 

E2 

X  SEX 

29 

ETHNICITY 

BY  GENDER  (BLACK  FEMALE) 

29 

E3 

X  SEX 

30 

ETHNICITY 

BY  GENDER  (HISPANIC  FEMALE) 

30 

E4 

X  SEX 

31 

ETHNICITY 

BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

31 

E2 

X  PE2 

32 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  HS  GPxAD) 

32 

E2 

X  PE3 

33 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  POST  HS) 

33 

E2 

X  PE4 

34 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

34 

E2 

X  PE_ 

35 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

35 

E3 

X  PE2 

36 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

36 

E3 

X  ?E3 

37 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

37 

E3 

X  PE4 

38 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

38 

E3 

X  PE_ 

39 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

E4 

X  PE2 

40 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

39 

E4 

X  PE3 

41 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

40 

E4 

X  PE4 

42 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

41 

E4 

X  PE_ 

43 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

42 

MA,<MG 

44 

MODAL  AGE 

,  LESS  THAN  MODAL  GRADE 
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Table  B.4 
(continued) 


VARIABLE 

LABEL 

DESCRIPTION 

MA,MG 

45 

MODAL  AGE,  MODAL  GRADE,  MISSING 

MA  >Mn 

46 

MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

'>MA  Mn 

47 

GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

48 

SCHOOL  TYPE  (NOT  PUBLIC) 

LI 

49 

FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

PRESCH" 

50 

WENT  TO  PRESCHOOL  (YES) 

49 

#PARENT1 

51 

SINGLE/MULTIPLE  PARENT  HOME  (MOTHER ,  FATHER  HOME) 

50 

MOTHER 

52 

MOTHER  AT  HOME 

51 

MOWORK 

53 

MOTHER  WORKS  OUTSIDE  (YES) 

52 

SCI123 

54 

SPENT  AT  LEAST  ONCE  A  WEEK  STUDYING  SCIENCE 

53 

SCI45 

55 

SPENT  <  ONCE.  A  WEEK  OR  NEVER  STUDYING  SCIENCE 

5'+ 

COMPUTER 

56 

USE  COMPUTERS  FOR  MATH,  READING,  ETC.  (YES) 

55 

SUPERVIS 

57 

ADULT  SUPERVISION  OF  STUDENT  AFTER  SCHOOL(YES) 

56 

MATH  Ql 

58 

MATH    QUANTILE  (LINEAR  -1,0,1) 

57 

SGI  Ql 

59 

SCIENCE    QUANTILE  (LINEAR  -1,0,1) 
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Table  B.5 


Estimated  Effects  for  Mathematics  Conditioning  Variables,  Grade  3/Age  9 


SUBSCALE    SUBSCALE  SUBSCALE 


3 

4 

5 

1 

-0. 

946172 

-1. 

218839 

-1. 

392913 

2 

-0. 

117644 

-0. 

055138 

0. 

001133 

3 

-0. 

237354 

-0. 

090877 

"0. 

273892 

4 

-0. 

230329 

-0. 

185513 

0. 

053999 

5 

-1. 

129916 

"1. 

191336 

-0. 

388513 

6 

0. 

173127 

0. 

153634 

0. 

177416 

7 

0. 

038667 

0. 

042726 

0. 

067788 

8 

0. 

000488 

0. 

052812 

0. 

089764 

9 

0. 

063241 

0. 

078478 

-0. 

046947 

10 

-0, 

D08171 

0. 

013199 

"0. 

012076 

11 

0. 

027463 

0. 

118840 

0. 

027394 

12 

0. 

167774 

0. 

271684 

0. 

255879 

13 

0. 

174402 

0. 

268882 

0. 

184006 

14 

0. 

059968 

0. 

214283 

0. 

129780 

15 

0. 

059607 

0. 

090701 

0. 

111263 

16 

0. 

131430 

0. 

134616 

0. 

164054 

17 

0. 

126861 

0, 

160665 

0. 

147785 

18 

-0, 

018847 

-0 

023137 

"0. 

022627 

19 

-0, 

032778 

0. 

272767 

0. 

421672 

20 

-0 

026220 

-0 

031712 

"0 

009785 

21 

-0, 

020154 

-0 

138456 

0 

051248 

22 

0 

339933 

0 

506240 

0 

330273 

23 

-0 

010525 

0 

122951 

-0 

012716 

24 

-0 

153557 

-0 

222950 

-0 

260143 

25 

-0 

056131 

-0 

044131 

-0 

036394 

26 

-0 

079226 

-0 

.030625 

-0 

010398 

27 

-0 

020886 

-0 

.028026 

-0 

002443 

28 

0 

.128932 

0 

.111914 

0 

106072 

29 

-0 

.032425 

-0 

.030307 

-0 

039899 

30 

-0 

.161538 

0 

.002850 

0 

.070964 

31 

-0 

.046617 

-0 

.183305 

-0 

.069007 

32 

-0 

.010292 

-0 

.132254 

"0 

.113742 

33 

-0 

.100050 

"0 

.279062 

-0 

.061269 

34 

-0 

.045574 

-0 

.247596 

0 

.002515 

35 

0 

.032819 

0 

.143784 

-0 

.232536 

36 

0 

.091916 

-0 

.010643 

-0 

.092038 

37 

0 

.022051 

-0 

.015300 

-0 

.191471 

38 

0 

.094694 

0 

.130126 

-0 

.150769 

39 

1 

.092026 

0 

.470541 

0 

.283398 

40 

0 

.616160 

0 

.926217 

0 

.045507 

41 

0 

.669838 

0 

.984378 

0 

.100432 

42 

0 

.967825 

1 

.193762 

0 

.375092 

43 

-0 

.617882 

"0 

.614600 

"1 

.001589 

44 

0 

.000186 

0 

.056692 

0 

.034156 

45 

0 

.318059 

0 

.350188 

0 

.509326 
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Table  B.5 
(continued) 


SUBSCALE    SUBSCALE  SUBSCALE 


46  -0.203138  -0.137937  -0.153053 

47  -0.045163  -0.050615  -0.096315 

48  0.013182  -0.015853  0.032462 

49  0.055456    0.108480  0.097324 

50  0.053331    0.095496  0.094727 

51  0.089618    0.006504  -0.024782 

52  -0.003609  -0.014105  0.033141 

53  -0.160670  -0.190548  -0.255744 

54  -0.181168  -0.244880  -0.305094 

55  0.006140  -0.010925  0.015374 

56  0.070747    0.023675  0.045945 

57  -0.188164  -0.199973  -0.187535 


3 


4 


5 
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Table  B.6 


Mathematics  Conaitioning  Variables,  Grade  7/Age  13 


VARIABLE 

LABEL 

DESCRIPTION 

1 

OVERALL 

1 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

3 

ETHNICITY 

(BLACK) 

4 

ETHNIC3 

4 

ETHNICITY 

(HISPANIC) 

5 

ETHNIC4 

5 

ETHNICITY 

(ASIAN  AMERICAN) 

6 

ST0C2 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

8 

REGION  (SOUTHEAST) 

9 

REGI0N3 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

10 

REGION  (WEST) 

11 

PARED2 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

ITEMS2 

15 

ITEMS  IN  HOME  (FOUR  OF  THE  FIVE) 

16 

ITEMS  3 

16 

ITEMS  IN  HOME  (FIVE  OF  THE  FIVE) 

17 

TV 

17 

HOURS  TV  WATCHING  (LINEAR) 

18 

TV**2 

18 

HOURS  TV  WATCHING  (QUADRATIC) 

19 

HW- 

•NO 

19 

HOMEWOPJK  (DON'T  HAVE  ANY) 

20 

HW- 

•YES 

20 

HOMEWORK  (YES  -  SOME  AMOUNT) 

21 

HW- 

•2345 

21 

HOMEWORK  AMOUNT  (LINEAR) 

22 

LM 

BY  E3 

22 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISPANIC) 

23 

LM 

BY  E4 

23 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  ASIAN  AMER) 

24 

LM 

BY  E_ 

24 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  OTHER  ETH) 

25 

LUNCHX 

25 

PERCENT  IN  LUNCH  PROGRAM 

26 

LUNCH_ 

26 

LUNCH  PROGRAM  (MISSING) 

27 

%WHITE49 

27 

PERCENT  WHITE  IN  SCHOOL  (0-49%  WHITE  MINORITY) 

28 

%WHITE79 

28 

PERCENT  WHITE  IN  SCHOOL  (50% -7 9%) 

XWHITEOO 

29 

PERCENT  WHITE  IN  SCHOOL  (80-100%  PREDOMINANTLY) 

29 

E2 

X  SEX 

30 

ETHNICITY 

BY  GENDER  (BLACK  FEMALE) 

30 

E3 

X  SEX 

31 

ETHNICITY 

BY  GENDER  (HISPANIC  FEMALE) 

31 

E4 

X  SEX 

32 

ETHNICITY 

BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

32 

E2 

X  PE2 

33 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

33 

E2 

X  PE3 

34 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  POST  HS) 

34 

E2 

X  PE4 

35 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,   COLLEGE  GRAD) 

35 

E2 

X  PE_ 

36 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

36 

E3 

X  PE2 

37 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

37 

E3 

X  PE3 

38 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

38 

E3 

X  PE4 

39 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

39 

E3 

X  PE_ 

40 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

^C 

E4 

X  PE2 

41 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

41 

E4 

X  PE3 

42 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

42 

E4 

X  PE4 

43 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

43 

E4 

X  PE_ 

44 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AM.ER,  UNKNOWN) 
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Table  B.6 
(continued) 


TT  A  D  T  A  HT  tr 

VARiAiSLr, 

T  ART7T 

HA,  <rlb 

y.  c 
45 

vr  A  vf/-' 

HA ,  nb 

40 

MA  ^>MP 

A  7 
4/ 

Aft 

AQ 
4y 

ACV"  OTJ? 

31 

://PAPFNTl 

^0 
3Z 

MnTVTPP 

nuwuivK 

RA 
34 

c  c 
D  J 

56 

MATH3 

57 

MATH45 

58 

SCIENCE2 

59 

SCIENCES 

60 

SCIENCE4 

61 

SCIENCES 

62 

GRADES 

63 

MATH  Ql 

64 

SCI  Ql 

DESCRIPTION 

45  MODAL  AGE,  LESS  THAN  MODAL  GRADE 

46  MODAL  AGE,  MODAL  GRADE,  MISSING 

47  MOLAL  AGE,  GREATER  THAN  MODAL  GRADE 

48  GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

49  SCHOOL  TYPE  (NOT  PUBLIC) 

50  FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

51  WENT  TO  PRESCHOOL  (YES) 

52  SINGLE/MULTIPLE  PARENT  HOME  (MOTHER, FATHER  HOME) 

53  MOTHER  AT  HOME 

54  MOTHER  WORKS  OUTSIDE  (YES) 

55  USE  COMPUTERS  FOR  MATH,  READING,  ETC,  (YES) 

56  TYPE  OF  MATH  CLASS  (REGULAR  MATH) 

57  TYPE  OF  MATH  CLASS  (PRE- ALGEBRA) 

58  TYPE  OF  MATH  CLASS (ALGEBRA) 

59  STUDYING  IN  SCIENCE  THIS  YEAR  (LIFE  SCIENCE) 

60  STUDYING  IN  SCIENCE  THIS  YEAR  (PHYSICAL  SCIENCE) 

61  STUDYII^G  IN  SCIENCE  THIS  YEAR  (EARTH  SCIENCE) 

62  STUDYING  IN  SCIENCE  THIS  YEAR  (GENERAL  SCIENCE) 

63  GRADES  IN  SCHOOL  (LINEAR) 

64  MATH    QUANTILE  (LINEAR  -1,0,1) 

65  SCIENCE    QUANTILE  (LINEAR  -1,0,1) 
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Table  B.7 


Estimated  Effects  for  Mathematics  Conditioning  Variables,  Grade  7/Age  13 


SUBSCALE    SUBSCALE    SUBSCALE  SUBSCALE 


3 

4 

5 

6 

1 

-0. 

352342 

-0. 

237241 

-0 

.232741 

-1. 

566806 

2 

-0. 

118358 

-0. 

121051 

0 

.027736 

-0. 

138846 

3 

-0. 

243163 

-0. 

241068 

-0 

.192695 

-0. 

157660 

4 

-0. 

143504 

-0. 

086834 

-0 

.087586 

0. 

001369 

5 

0. 

154143 

-0. 

123063 

0 

.329942 

-0. 

345517 

6 

0. 

106950 

0. 

043702 

0 

.072136 

-0. 

007275 

7 

0. 

041103 

0. 

005016 

0 

.027964 

0. 

002061 

8 

-0. 

101002 

-0. 

076939 

-0 

.068503 

-0. 

121620 

9 

-0. 

019602 

0. 

003319 

-0 

.066997 

-0. 

101269 

10 

-0. 

133251 

-0. 

104762 

-0 

.067039 

-0. 

239011 

11 

-0. 

015131 

0. 

076167 

-0 

.039755 

0. 

118204 

12 

0. 

109763 

0. 

215349 

0 

.087665 

0. 

247897 

13 

0. 

149055 

0. 

215896 

0 

.092551 

0. 

259444 

14 

-0. 

053523 

-0. 

008094 

-0 

.054022 

0. 

173222 

15 

0. 

082457 

0. 

020266 

0 

.062806 

0. 

085506 

16 

0. 

065393 

0. 

048085 

0 

.074417 

0. 

161452 

17 

0. 

073241 

0. 

057704 

0 

.078217 

0. 

032170 

18 

-0. 

014776 

-0. 

011898 

-0 

.010180 

-0. 

009676 

19 

-0. 

147903 

-0. 

230917 

-0 

.150158 

0. 

055594 

20 

-0. 

080464 

-0. 

141038 

-0 

.087860 

0. 

117139 

21 

-0. 

006627 

-0. 

024537 

-0 

.000802 

-0. 

008386 

22 

0. 

058210 

0. 

017234 

0 

.072507 

-0. 

002178 

23 

0. 

061216 

-0. 

023127 

-0 

.049094 

-0. 

273219 

24 

-0. 

090200 

-0. 

051471 

-0 

.086291 

0. 

030243 

25 

-0. 

098591 

-0. 

070908 

-0 

.101800 

-0. 

432306 

26 

-0. 

006329 

-0. 

008319 

-0 

.050810 

-0. 

061885 

27 

-0. 

073191 

-0. 

042237 

-0 

.111685 

-0. 

027993 

28 

-0. 

002039 

-0. 

006013 

-0 

.015316 

0. 

084651 

29 

0. 

001871 

0. 

016668 

-0 

.064368 

0. 

045342 

30 

0. 

027987 

0. 

017813 

0 

.009011 

-0. 

025125 

31 

-0. 

183716 

-0. 

048277 

-0 

.239876 

0. 

125414 

32 

0. 

0560^9 

0. 

004230 

0 

.187129 

-0. 

033398 

33 

-0. 

013261 

0. 

014572 

0 

.096781 

-0. 

261570 

34 

-0. 

134803 

-0. 

105469 

-0 

.010082 

-0. 

231216 

35 

0. 

079497 

0. 

026672 

0 

.088876 

-0. 

187968 

36 

0. 

013273 

-0. 

019971 

0 

.018582 

0. 

009769 

37 

0. 

007303 

0. 

024266 

-0 

.092669 

-0. 

234611 

38 

-0. 

058938 

-0. 

063571 

-0 

.045748 

-0. 

058699 

39 

-0. 

011958 

-0. 

024379 

0 

.054089 

-0. 

053586 

40 

0. 

023058 

0. 

089013 

-0 

.511295 

-0. 

484603 

41 

0. 

132417 

0. 

296604 

-0 

.068088 

0. 

211198 

42 

-0. 

021448 

0. 

395627 

0 

.022164 

0. 

878078 

43 

0. 

043843 

0. 

513521 

0 

.048291 

0. 

527336 

44 

-0. 

351336 

-0. 

447323 

-0 

.4296^1 

-0. 

283039 

45 

0. 

015146 

-0. 

072972 

0 

.002604 

0. 

099541 
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Table  B.7 
(continued) 


SUBSCALE    SUBSCALE    SUBSCALE  SUBSCALE 


A 

5 

6 

4o 

u . 

0 

0. 

310623 

0. 

454944 

/.  "7 

-  u . 

1 7091 n 

-0 
"  u  . 

-0. 

155813 

-0. 

062705 

A  O 

"  u . 

AO  /.  7  Q  0 
UZh / o  Z 

A 
U  . 

0 

008822 

\J\J  \j  KJ 

0. 

020517 

A 

-  u . 

A^7/C1  0 
UO / OlZ 

A 

*•  U  . 

yjoo  £.  1. 1. 

-0 

051669 

-0. 

079223 

C  A 

50 

A 

A  Q  'J  A  Q  /. 

A 
U  . 

n 
\j . 

yj  1.  D  y 

0. 

039496 

51 

A 
U  . 

A/i  1  C  Q  Q 

A 
U  . 

m  f^^  LQ 

0 

007752 

0. 

041884 

CO 

A 

-  U  . 

A 
•  U  . 

0 . 

017938 

0. 

062273 

D  J 

A 
U  . 

UU t  too 

~  u  . 

-0. 

006762 

-0. 

053299 

C  A 

54 

A 

-  U  . 

A7  Rl  1  'J 

A 

-  U  . 

UO       7  D 

.0 

068931 

-0 

119231 

J  J 

n 

u 

n7sn7n 

0 

101699 

0 

.001479 

0 

046058 

56 

0 

,262637 

0 

.290710 

0 

.199434 

0 

.311969 

57 

0 

.129170 

0 

.211722 

0 

.021191 

0 

.089921 

58 

0 

.071305 

0 

.052191 

0 

.106246 

0 

.127636 

59 

0 

.084178 

0 

.009127 

0 

.036155 

0 

.056185 

60 

0 

.061501 

0 

.008557 

0 

.078339 

0 

.049026 

61 

0 

.117946 

0 

.066933 

0 

.079968 

0 

.057133 

62 

0 

.225022 

0 

.253259 

0 

.227615 

0 

.306723 

63 

-0 

.145286 

-0 

.196210 

-0 

.118398 

-0 

.193678 

64 

-0 

.202314 

-0 

.177616 

-0 

.169222 

-0 

.270378 
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Table  B.8 

Mathematics  Conditioning  Variables,  Grade  11/Age  17 


VARIABLE 

LABEL  DESCRIPTION 

1  OVERALL  1  OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2  GENDER2  2  SEX  (FEMALE) 

3  ETHNIC2  3  ETHNICITY  (BLACK) 

4  ETHNIC3  4  ETHNICITY  (HISPANIC) 

5  ETHNIC4  5  ETHNICITY  (ASIAN  AMERICAN) 

6  ST0C2  6  SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7  ST0C3  7  SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8  REGI0N2  8  REGION  (SOUTHEAST) 

9  REGI0N3  9  REGION  (CENTRAL) 

10  REGI0N4  10  REGION  (WEST) 

11  PARED2  11  PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12  PARED3  12  PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13  PARED4  13  PARENTS  EDUCATION  (COLLEGE  GRAD) 

14  PARED_  14  PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15  ITEMS2  15  ITEMS  IN  HOME  (FOUR  OF  THE  FIVE) 

16  ITEMS3  16  ITEMS  IN  HOME  (FIVE  OF  THE  FIVE) 
1-7  TV  17  HOURS  TV  WATCHING  (LINEAR) 

18  TV**2  18  HOURS  TV  WATCHING  (QUADRATIC) 

19  HW-NO  19  HOMEWORK  (DON'T  HAVE  ANY) 

20  HW-YES  20  HOMEWORK  (YES  -  SOME  AMOUNT) 

21  HW-2345  21  HOMEWORK  AMOUNT  (LINEAR) 

22  LM  BY  E3  22  LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISPANIC) 

23  LM  BY  E4  23  LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  ASIAN  AMER) 

24  LM  BY  E_  24  LANGUAGE  MINORITY  BY  ETHNICITY  (YES,   OTHER  ETH) 

25  LUNCH%  25  PERCENT  IN  LUNCH  PROGRAM 

26  LUNCH_  26  LUNCH  PROGRAM  (MISSING) 

27  %WHITE49  27  PERCENT  WHITE  IN  SCHOOL  (0-49%  WHITE  MINORITY) 

28  5:WHITE79  28  PERCENT  WHITE  IN  SCHOOL  (50-79%  INTEGRATED) 
XWHITEOO  29  PERCENT  WHITE  IN  SCHOOL  (80-100%  PREDOMINANTLY) 

29  E2  X  SEX  30  ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

30  E3  X  SEX  31  ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

31  E4  X  SEX  32  ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

32  E2  X  PE2  33  ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

33  E2  X  PE3  34  ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

34  E2  X  PE4  35  ETHNICITY  BY  PARENT'S  ED  (BIACK,  COLLEGE  GRAD) 

35  E2  X  PE_  36  ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

36  E3  X  PE2  37  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

37  E3  X  PE3  38  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

38  E3  X  PE4  39  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

39  E3  X  PE_  40  ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

40  E4  X  PE2  41  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

41  E4  X  PE3  42  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,   POST  HS) 

42  E4  X  PE4  43  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 
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Table  B.8 
(continued) 


\7AR  TART.F 

LABEL 

E4  X  PE 

LL 

MA  <rMn 

MA  Mn 

MA  >Mn 

LI 

>MA  MG 

M-o 

cpu  TYPE 

ZlQ 

PRFSCHl 

J  JL 

MOTHFR 

MOUORK 

GRADES 

55 

PR0GRAM2 

56 

PROGRAMS 

57 

MATH 

58 

SCIENCE 

59 

F0STSEC2 

60 

P0STSEC3 

61 

HRS  WORK 

62 

ENGL23 

63 

ENGLISH5 

64 

MATH  Ql 

65 

SCI  Ql 

DESCRIPTION 

44  ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

45  MODAL  AGE,  LESS  THAN  MODAL  GRADE 

46  MODAL  AGE.  MODAL  GRADE,  MISSING 

47  MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

48  GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

49  SCHOOL  TYPE  (NOT  PUBLIC) 

50  FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

51  WENT  TO  PRESCHOOL  (YES) 

52  SINGLE/MULTIPLE  PARENT  HOME  (MOTHER, FATHER  HOME) 

53  MOTHER  AT  HOME 

54  MOTHER  WORKS  OUTSIDE  (YES) 

55  GRADES  IN  SCHOOL  (LINEAR) 

56  HIGH  SCHOOL  PROGRAM (COLLEGE  PREPARATORY) 

57  HIGH  SCHOOL  PROGRAM (VOCATIONAL) 

58  NUMBER  OF  MATH  COURSES 

59  NUMBER  OF  SCIENCE  COURSES 

60  TWO-YEAR  COLLEGE 

61  FOUR -YEAR.  COLLEGE 

62  HOURS  OF  OUTSIDE  WORK 

63  TYPES  OF  ENGLISH  CLASS (ADVANCED&COLLEGE  PREP.) 

64  TYPES  OF  ENGLISH  CLASS (REMEDIAL) 

65  MATH    QUANTILE  (LINEAR  -1.0,1) 

66  SCIENCE    QUANTILE  (LINEAR  -1,0,1) 


-478- 


ERLC 


530 


Table  B.9 


Estimated  Effects  for  Mathematics 


SUBSCALE    SUBSCALE  SUBSCALE 


3 

4 

5 

1 

0. 

195858 

0 

304516 

0 

.142497 

2 

-0. 

246885 

-0 

193048 

-0 

.021336 

3 

-0. 

344944 

-0. 

270933 

-0 

160073 

4 

-0. 

191312 

-0. 

137035 

-0 

.171029 

5 

0. 

294510 

-0 

084349 

0 

215611 

6 

0. 

215009 

0, 

151762 

0 

135865 

7 

0. 

137500 

0, 

050463 

0 

063095 

8 

-0. 

052084 

-0 

016224 

0 

.041018 

9 

0. 

072164 

0 

053202 

0 

017236 

10 

-0. 

010277 

-0 

023525 

-0 

012635 

11 

0. 

018629 

0 

018233 

-0 

.041512 

12 

0. 

094391 

0 

056743 

0 

.004418 

13 

0. 

066504 

0, 

053733 

-0 

023865 

14 

-0. 

229595 

-0 

080071 

-0 

.132718 

15 

0. 

041714 

0 

046100 

0 

063375 

16 

0. 

056468 

0 

060345 

0 

041185 

17 

-0. 

006456 

0 

015476 

0 

019801 

18 

-0. 

003765 

-0 

007944 

-0 

005793 

19 

-0. 

317430 

-0 

175180 

-0 

133841 

20 

-0. 

152795 

0 

001618 

-0 

025344 

21 

-0. 

027925 

-0, 

024972 

-0 

005854 

22 

0. 

043355 

0, 

030421 

0 

021115 

23 

-0. 

167295 

-0 

190842 

-0 

.167424 

24 

-0. 

095753 

-0 

061741 

-0 

043180 

25 

-0. 

118837 

-0. 

109148 

-0 

005480 

26 

-0. 

028970 

-0 

037728 

-0 

023312 

27 

-0. 

032063 

-0 

006945 

-0 

038716 

28 

-0. 

001481 

0 

029177 

0 

023390 

29 

0. 

069319 

0 

044137 

0 

064818 

30 

-0. 

042455 

0 

038289 

0 

064913 

31 

0. 

287757 

-0. 

054389 

-0 

022413 

32 

-0. 

129335 

-0. 

100427 

0 

017015 

33 

-0. 

213154 

-0 

060476 

-0 

108509 

34 

-0. 

104558 

-0 

062576 

-0 

054697 

35 

0. 

166319 

0 

015014 

0 

170531 

36 

-0. 

060255 

-0 

082799 

0 

045010 

37 

0. 

001950 

-0 

086444 

-0 

052552 

38 

0. 

038466 

0 

023252 

0 

038175 

39 

0. 

192027 

-0 

045704 

0 

111945 

40 

-0. 

248807 

0 

041465 

-0 

.068258 

41 

-0. 

265656 

0 

359645 

-0 

.133069 

42 

-0. 

132709 

0 

349036 

0 

100124 

43 

-0. 

013802 

0 

236105 

0 

174070 

44 

-0. 

133187 

-0 

216247 

-0 

.122409 

45 

-0. 

010745 

-0 

006971 

-0 

.015074 

Conditioning  Variables,  Grade  11/Age  17 

SUBSCALE  SUBSCALE 
6  7 


-0 

541098 

-1. 

119437 

-0 

260969 

-0. 

073420 

-0 

470358 

-0. 

260151 

-0 

063208 

-0. 

114471 

0 

411057 

0. 

182352 

0 

148188 

0. 

208643 

0 

042328 

0. 

039243 

-0 

068872 

-0. 

002903 

0 

010793 

0. 

038770 

-0 

062159 

-0. 

086379 

0 

035876 

-0. 

062526 

0 

146590 

0. 

037264 

0 

175210 

0. 

017065 

-0 

097598 

-0. 

179930 

0 

068467 

0. 

063649 

0 

062993 

0. 

079294 

0 

030363 

-0. 

014262 

-0 

010510 

-0. 

003715 

-0 

490315 

-0. 

342579 

-0 

161060 

-0. 

227831 

-0 

006740 

0. 

002900 

0 

093909 

-0. 

006710 

-0 

335473 

0. 

025287 

-0 

112555 

-0. 

007060 

-0 

185414 

-0. 

258239 

0 

006773 

-0. 

003202 

-0 

073542 

0. 

036574 

0 

012689 

0. 

060631 

0 

036009 

0. 

030742 

-0 

109547 

0. 

004549 

0 

157593 

-0. 

160585 

-0 

017924 

0. 

023126 

-0 

007431 

-0. 

178859 

-0 

128184 

-0. 

071275 

0 

046204 

-0. 

074330 

-0 

084824 

0. 

023171 

-0 

035716 

-0. 

145061 

-0 

201052 

-0. 

070557 

0 

059751 

0. 

128328 

-0 

381340 

0. 

124824 

-0 

478293 

0. 

049367 

-0 

193004 

0. 

067695 

0 

007380 

0. 

160958 

-0 

253036 

-0. 

230837 

-0 

050555 

-0. 

040595 
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Table  B.9 
(continued) 


SUBSCALE    SUBSCALE    SUBSCALE    SUBSCALE  SUBSCALE 


3 

4 

5 

6 

7 

46 

-0. 

020861 

0. 

023623 

-0. 

026568 

-0. 

182436 

-0. 

090257 

47 

-0. 

114376 

-0. 

141685 

-0. 

129027 

-0. 

269012 

-0. 

292650 

48 

-0. 

046394 

-0. 

093190 

-0. 

019327 

-0. 

077206 

-0. 

019410 

49 

-0. 

043236 

-0. 

030714 

-0. 

006119 

-0. 

052606 

-0. 

051349 

50 

0. 

003831 

0. 

000824 

0. 

011364 

0. 

030711 

0. 

035063 

51 

0. 

036420 

0. 

002728 

-0. 

000217 

-0. 

003587 

-0. 

006095 

52 

-0. 

088439 

-0. 

066550 

-0. 

042036 

-0. 

143390 

-0. 

106678 

53 

0. 

038097 

0 

004660 

0. 

004990 

0. 

071296 

0. 

053183 

54 

0, 

197311 

0 

148064 

0 

135622 

0 

248827 

0 

252311 

55 

0 

092162 

0 

077809 

0 

054228 

0 

143756 

0 

198307 

56 

0 

040549 

0 

.001035 

-0 

.033087 

0 

.015966 

-0 

.056789 

57 

0 

.117385 

0 

.122233 

0 

.113171 

0 

.229784 

0 

.237471 

58 

0 

.056018 

0 

.044501 

0 

.019986 

0 

. 040499 

0 

.014109 

59 

-0 

.059698 

0 

.001888 

0 

.014762 

-0 

.034046 

0 

.022275 

60 

0 

.057070 

0 

.112711 

0 

.086981 

0 

.136935 

0 

.199039 

61 

-0 

.027140 

-0 

. 048412 

-0 

.077096 

-0 

.080121 

-0 

.086530 

62 

0 

.082593 

0 

.063262 

0 

.059468 

0 

.113782 

0 

.140638 

63 

-0 

.077388 

-0 

.132642 

-0 

.012501 

-0 

.185554 

-0 

. 020404 

64 

-0 

.175084 

-0 

.203456 

-0 

.123231 

-0 

.195708 

-0 

.156517 

65 

-0 

.258420 

-0 

.198467 

-0 

.154683 

-0 

. 296144 

-0 

.233239 
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Table  B,10 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1977-78:    Age  9 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-0. 

273301 

OVERALL 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

0. 

026875 

GENDER2 

SEX  (FEMALE) 

3 

-0, 

705243 

ETHNIC 2 

OBSERVED  ETHNICITY  (BLACK) 

4 

-0. 

218671 

ETHNIC3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

299975 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

453148 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0, 

210676 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

304078 

REGI0N2 

REGION  (SOUTHEAST) 

9 

-0. 

018972 

REGI0N3 

REGION  (CENTRAL) 

10 

-0. 

267138 

REGI0N4 

REGION  (WEST) 

11 

0. 

283547 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0 

461085 

PARED3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0 

444294 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

0 

123491 

PARED_ 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-0 

899835 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0 

466793 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0 

.297460 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

0 

.445589 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

0 

.139320 

E2  X  SEX 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

20 

-0 

.110746 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

-0 

.094134 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

-0 

.026872 

E2  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

-0 

.165483 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

24 

-0 

.014514 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

25 

0 

.047939 

E2  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

0 

.290944 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

27 

-0 

.095222 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

28 

-0 

.275784 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

0 

.090747 

E3  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

-0 

.090926 

E4  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,   COLL  GRAD) 

31 

0 

.018214 

E4  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

32 

0 

.010844 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 
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Table  B.ll 

Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1977-78:    Age  13 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

1 

-0.506104 

OVERALL 

2 

-0.058437 

GE1NDER2 

3 

-0.757338 

ETHNIC2 

4 

-0.329396 

ETHNIC3 

5 

-0.106764 

ETHNIC4 

6 

0.475734 

ST0C2 

7 

0.264444 

ST0C3 

8 

-0.334031 

REGI0N2 

9 

0.008634 

REGI0N3 

10 

-0.205949 

REGI0N4 

11 

0.273782 

PARED 2 

12 

0.507748 

PARED 3 

13 

0.693130 

PARED4 

14 

-0.011754 

PARED_ 

15 

-0.760385 

<  MODALG 

16 

0.782461 

>  MODALG 

17 

0.269945 

H0MEITM3 

18 

0.441897 

H0MEITM4 

19 

0.193220 

E2  X  SEX 

20 

0.159007 

E3  X  SEX 

21 

-0.137822 

E4  X  SEX 

22 

-0.186209 

E2  X  PE2 

23 

-0.233521 

E2  X  PE3 

24 

-0.359043 

E2  X  PE4 

25 

-0.175012 

E2  X  PE_ 

26 

-0.150425 

E3  X  PE2 

27 

-0.174618 

E3  X  PE3 

28 

-0.244380 

E3  X  PE4 

29 

-0.209913 

E3  X  PE_ 

30 

0.847664 

E4  X  PE4 

31 

0.252726 

E4  X  PE_ 

32 

0.036806 

S-.H.PRIV 

DESCRIPTION 
OVERALL  CONSTANT  '1'  FOR  EVERYONE 
SEX  (FEMALE) 

OBSERVED  ETHNICITY  (BLACK) 
OBSERVED  ETHNICITY  (HISPANIC) 
OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 
SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 
SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 
REGION  (SOUTHEAST) 
REGION  (CENTRAL) 
REGION  (WEST) 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 
PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 
PARENTS  EDUCATION  (COLLEGE  GRAD) 
PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 
MODAL  GRADE  (LESS  THi^N  MODAL  GRADE) 
MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 
ARTICLES  IN  HOME  (YES  TO  3) 
ARTICLES  IN  HOME  (YES  TO  4) 
ETHNICITY  BY  GENDER  (BLACK  FEMALE) 
ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 
ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 
ETHNICITY  BY  PARENT'S  ED  (BL^CK,  HS  GRAD) 


ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 
ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

ED  (BLACK,  UNKNOWN) 
ED  (HISPANIC,  HS  GRAD) 


ETHNICITY  BY  PARENT'S 
ETHNICITY  BY  PARENT'S 


ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 
ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 
ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 
ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 
ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 
SCHOOL  TYPE  (PRIVATE) 
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Table  B.12 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 


1977-78:    Age  17 

ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-0. 

781467 

OVERALL 

OVERALL  CONSTANT  '1'  FOR 

EVERYONE 

2 

-0. 

206773 

GENDER2 

SEX  (FEMALE) 

3 

-0. 

431460 

ETHNIC2 

OBSERVED  ETHNICITY  (BLACK) 

4 

-0. 

005593 

ETHNIC3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

134567 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

352374 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0. 

221161 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

066068 

REGI0N2 

REGION  (SOUTHEAST) 

9 

0. 

058216 

REGI0N3 

REGION  (CENTRAL) 

10 

-0. 

042895 

REGI0N4 

REGION  (WEST) 

11 

0. 

084179 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0. 

199708 

PARED3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0. 

305552 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

-0. 

040135 

PARED_ 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-0. 

363890 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0. 

078896 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0. 

178779 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO 

3) 

18 

0. 

22151 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

-0. 

053206 

E2  X  SEX 

ETHNICITY  LY  GENDER  (BLACK  FEMALE) 

20 

-0, 

285367 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

0, 

090859 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

-0 

174525 

E2  X  PE2 

ETHNICITY  BY  PARENT'S  ED 

(BLACK,  HS  GRAD) 

23 

-0 

065219 

22  X  PE3 

ETHNICITY  BY  PARENT'S  ED 

(BLACK,  POST  HS) 

24 

-0 

.225030 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED 

(BLACK,  COLLEGE  GRAD) 

25 

-0 

140388 

E2  X  PE_ 

ETHNICITY  BY  PARENT'S  ED 

(BLACK,  UNKNOWN) 

26 

-0 

.066370 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED 

(HISPANIC,  HS  GRAD) 

27 

-0 

.189053 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED 

(HISPANIC,  POST  HS) 

28 

-0 

.140345 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED 

(HISPANIC,  COLLEGE) 

29 

-0 

.319494 

E3  X  PE_ 

ETHNICITY  BY  PARENT'S  ED 

(HISPANIC,  UNKNOWN) 

30 

-0 

.429663 

E4  X  PE2 

ETHNICITY  BY  PARENT'S  ED 

(ASIAN  AMER,  HS  GRAD) 

31 

0 

.506253 

E4  X  PE3 

ETHNICITY  BY  PARENT'S  ED 

(ASIAN  AMER,  POST  HS) 

32 

0 

.105962 

E4  X  PE4 

ETHNICITY  BY  PARENT'S  ED 

(ASIAN  AMER,  COLL  GRAD) 

33 

-0 

.038407 

SCH . PRIV 

SCHOOL  TYPE  (PRIVATE) 

34 

0 

.026481 

TV. 0-2 

TV  WATCHING  (0-2  HOURS) 

35 

-0 

.029427 

TV. 3-5 

TV  WATCHING  (3-5  HOURS) 

36 

-0 

.155838 

TV.6+ 

TV  WATCHING  (6+  HOURS) 

37 

-0 

.205242 

HW-NO 

HOMEWORK  (NONE  ASSIGNED) 

38 

-0 

.128222 

HW-YES 

HOMEWORK  (YES  -  SOME  AMOUNT) 

39 

0 

.045070 

HW-2345 

HOMEWORK  AMOUNT  (LINEAR) 

40 

-0 

.211903 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (OFTEN) 

41 

-0 

.060369 

H0MELNG2 

OTHER  LAl^GUAGE  AT  HOME  (SOMETIMES) 

42 

-0 

.087101 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

43 

-0 

.167821 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

44 

0 

.158570 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

45 

-0 

.117209 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 
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Table  B.12 
(continued) 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

46 

-0.191573 

NMATHl 

47 

0,228896 

NMATH2 

48 

0.682312 

NMATH3 

49 

1.029129 

NMATH4 

50 

1.321908 

NMATH5 

51 

0.076033 

COMPUTER 

DESCRIPTION 
HIGHEST  LEVEL  MATH  TAKEN  (PRE-ALGEBRA) 
HIGHEST  LEVEL  MATH  TAKEN  (ALGEBRA) 
HIGHEST  LEVEL  MATH  TAKEN  (GEOMETRY) 
HIGHEST  LEVEL  MATH  TAKEN  (ALGEBRA- 2) 
HIGHEST  LEVEL  MATH  TAKEN  (CALCULUS) 
COMPUTER  CLASS  TAKEN  (YES) 
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Table  B.13 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 


1981-82:    Age  9 

ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-0. 

862638 

OVERALL 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

0. 

006737 

GENDER2 

SEX  (FEMALE) 

3 

-0. 

363383 

ETHNIC2 

OBSERVED  ETHNICITY  (BLACK) 

4 

-0. 

252489 

ETHNIC3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

226445 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

497516 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0. 

177121 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

245979 

REGI0N2 

REGION  (SOUTHEAST) 

9 

-0. 

124888 

REGI0N3 

REGION  (CENTRAL) 

10 

-0. 

150039 

REGI0N4 

REGION  (WEST) 

11 

0 

258322 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0, 

446228 

PARED3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0, 

451693 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

0 

241492 

PARED_ 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-0 

875036 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0 

.662891 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0 

193038 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

0 

.315100 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

-0 

.042492 

E2  X  SEX 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

20 

-0 

.053017 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

0 

.072069 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

-0 

.049272 

E2  X  PE2 

"ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

-0 

.220469 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,   POST  HS) 

24 

-0 

.200338 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,   COLLEGE  GRAD) 

25 

-0 

.167562 

E2  X  PE_ 

ETHNICITY  3Y  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

-0 

.138775 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  KS  GRAD) 

27 

-0 

.355953 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,   POST  HS) 

28 

-0 

.350525 

E3  X  PE4 

ETHNICI'/Y  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

-0 

.300426 

E3  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

0 

.079773 

E4  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,   COLL  GRAD) 

31 

-0 

.048421 

E4  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

32 

0 

.103982 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 

33 

0 

.648663 

TV. 0-2 

TV  WATCHING  (0-2  HOURS) 

34 

0 

.831681 

TV. 3-5 

TV  WATCHING  (3-5  HOURS) 

35 

0 

.648024 

TV.6+ 

TV  WATCHING  (6+  HOURS) 

36 

-0 

.137906 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (OFTEN) 

37 

0 

.008072 

H0MELNG2 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES) 

38 

-0 

.075129 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

39 

-0 

.142182 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

40 

0 

.332070 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

41 

0 

.156723 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

ERIC 
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Table  B.14 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1981-82:    Age  13 


TTQTTMATFri 

VAR TABLE 

iLr  r  iLv^  1 

T  ARFT 

i_4£iiJ  ill  i-< 

DESCRIPTION 

1 

1    0  ^^00  ^ 
- i  .  Z jOZZ 3 

n\717PAT  T 

OVERATT   CONSTANT  '1'   FOR  EVERYONE 

2 

n  171  ft^n 
- U . i / iooU 

L7riINUrjK.Z 

cpy  ^FFMATF'i 

3 

T7T14'MTr'9 
xLlnlNlUZ 

ORSFRVFT)  ETHNICITY  (BLACK) 

4 

n   o  ^  Q  o  1  ^ 

T7T14XITr'7 

ORSFRVFD  ETHNICITY  (HISPANIC) 

5 

n    n  O  A  Q  A  o 

T7T14XITr'/i 

rjirlIN±U'+ 

nR<?FRVFn  ETHNICITY  (ASIAN  AMERICAN) 

c 
o 

O    A  1  n  1  Q 1 

QTnr'9 

^  lUL^Z 

ST7F  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

n   1  ft  SZtft'^ 

U  .  i.  O  -3'+0  J 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

o 
o 

n  91  '^R'^n 

~U  .  Z-LjOjU 

REGION  (SOUTHEAST) 

Q 

n  nn  71 1 

REGION  (CENTRAL) 

10 

n    1  1  Q  ^  Q  A 

RFfiTON  (UFST) 

11 

u . u jy4iz 

"DAD  trno 

PARFMT^  FDUCATTON  (HIGH  SCHOOL  GRAD) 

12 

u . z4 Jiby 

PAP  T7r»7 

PARFNT^  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PAP  T7r»/i 

PARFNTc;  FDUCATTON  (COLLEGE  GRAD) 

14 

U . U4oDzo 

PAPtrn 

PARFMTc:  FDUCATTON  (MISSING     I  DON'T  KNOW) 

15 

-0 . 644180 

^   MOHAT  r* 
<  nUUAL.b 

MonAT   PRAnF  (TFSS  THAN  MODAL  GRADE) 

16 

n    "7    o    n  o 

0 . /0809o 

MonAT   PPAHF  /'HRFATFR  THAN  MODAL  GRADE) 

1  "7 

17 

n    1  ^  Q  O  "71 

0 . iDoz  / 1 

nunrii  in.  J 

ARTTrTF<?   IN  HOME   (YES  TO  3) 

1  o 

18 

n     O  Q  Q  A  "7  "7 
0 . ZO J4 / / 

nUrirjl  in.'* 

ARTTCTF^?  IN  HOME   (YES  TO  4) 

ly 

U . UOO  iZ 7 

T?9  Y  <5T?y 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

-  U  .  Kjy  /  ZZ  / 

iL  J    A    i3 1-i  A 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

zl 

n   A1 070^ 
U . UiZ / ZD 

FA.  Y  <5FY 

rj'+    A    iJ  E*A 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

0  0 

U  .  UOU /  7 J 

F9  Y  PF9 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

-0  177f>9fi 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

9S 

E2  X  PE 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

Z.O 

-D  m9S9ft 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

97 

-0  nzi.'^'^9^ 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

9R 

-D  17QQfift 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

9Q 

-0  17^.nQS 

U.  X/*+L/7-^ 

E3  X  PE 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

'^n 

n   1  Q'^1  70 

F4  y  PE4 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

'^1 
^  1 

-.0  niA.QA.7 

F4  y  PE 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

J/ 

U  . U  JO  3UU 

<?rH  PR  TV 

SCHOOL  TYPE  (PRIVATE) 

- U . lUlZ J J 

T\7  0-9 
i  V  .  u  **  z, 

TV  WATCHING  (0-2  HOURS) 

34 

A    1  1  "7  O  ^  ^ 

-0 . li/zjj 

i  V  .  J  **  3 

T\7  UATrHTNn   C^-S  HOURS) 

0    99ft1 A7 
**U  .  ZZOXo  / 

TV  f>+ 

XV.  o  • 

TV  WATCHING  (6+  HOURS) 

0 . 231025 

HW-NO 

HOMEWORK  (NONE  ASSIGNED) 

37 

0.444984 

HW-YES 

HOMEWORK  (YES  -  SOME  AMOUNT) 

38 

-0.032439 

HW-2345 

HOMEWORK  AMOUNT  (LINEAR) 

39 

-0.227167 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (OFTEN) 

40 

-0.022530 

H0MELNG2 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES) 

41 

0.073600 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

42 

0.039274 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

43 

0.279979 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

44 

0.133273 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

45 

0.360884 

GRADES 

GRADES  IN  SCHOOL 
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Table  B.15 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1981-82:    Age  17 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

1 

-1. 

583301 

OVERALL 

2 

-0. 

271300 

GENDER2 

3 

-0. 

537188 

ETHNIC 2 

4 

-0. 

314424 

ETHNIC3 

5 

0. 

274670 

ETHNIC4 

6 

0. 

364567 

ST0C2 

7 

0. 

199603 

ST0C3 

8 

-0. 

027483 

REGI0N2 

9 

0. 

041125 

REGI0N3 

10 

-0. 

090060 

REGI0N4 

11 

0. 

104941 

PARED2 

12 

0. 

178396 

PARED 3 

13 

0. 

229555 

PARED4 

14 

-0. 

056325 

PAI^D_ 

15 

-0. 

224271 

<  MODALG 

16 

0. 

019933 

>  MODALG 

17 

0. 

068540 

H0MEITM3 

18 

0. 

100033 

H0MEITM4 

19 

0. 

137157 

E2 

X  SEX 

20 

0. 

046145 

E3 

X  SEX 

21 

0. 

153455 

E4 

X  SEX 

22 

-0. 

079989 

E2 

X  PE2 

23 

-0 

038142 

E2 

X  PE3 

24 

-0 

037173 

E2 

X  PE4 

25 

-0 

067290 

E2 

X  PE_ 

26 

-0 

012681 

E3 

X  PE2 

27 

0 

.073736 

E3 

X  PE3 

28 

0 

.002859 

E3 

X  PE4 

29 

-0 

.131598 

E3 

X  PE_ 

30 

-0 

.117925 

E4 

X  PE2 

31 

-0 

.169627 

E4 

X  PE3 

32 

-0 

.109758 

E4 

X  PE4 

33 

-0 

.116926 

SCH.PRIV 

34 

0 

.170632 

TV 

.0-2 

35 

0 

.101777 

TV 

.3-5 

36 

0 

.003121 

TV 

.6+ 

37 

0 

.041748 

HW 

-NO 

38 

0 

.236337 

HW 

-YES 

39 

-0 

.023902 

HW 

-2345 

40 

-0 

.165789 

HOMELNGl 

41 

-0 

.001730 

H0MELNG2 

42 

0 

.086177 

HL2  X  E2 

43 

-0 

.185237 

HLl  X  E2 

44 

0 

.211456 

HL2  X  E3 

45 

-0 

.050424 

HLl  X  E3 

DESCRIPTION 
OVERALL  CONSTANT  '1'  FOR  EVERYONE 
SEX  (FEMALE) 

OBSERVED  ETHNICITY  (BLACK) 
OBSERVED  ETHNICITY  (HISPANIC) 
OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 
SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 
SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 
REGION  (SOUTHEAST) 
REGION  (CENTRAL) 
REGION  (WEST) 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

ARTICLES  IN  HOME  (YES  TO  3) 

ARTICLES  IN  HOME  (YES  TO  4) 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

SCHOOL  TYPE  (PRIVATE) 

TV  WATCHING  (0-2  HOURS) 

TV  WATCHING  (3-5  HOURS) 

TV  WATCHING  (6+  HOURS) 

HOMEWORK  (NONE  ASSIGNED) 

HOMEWORK  (YES  -  SOME  AMOUNT) 

HOMEWORK  AMOUNT  (LINEAR) 

OTHER  LANGUAGE  AT  HOME  (OFTEN) 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES) 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 
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Table  B.15 
(continued) 


VARIABLE 

FFFFCT 

lABEL 

46 

-0.125447 

NMATHl 

47 

0.244314 

NMATH2 

48 

0.443787 

NMATH3 

49 

0.744452 

NMATH4 

50 

0.929301 

NMATH5 

51 

0.017961 

COMPUTER 

52 

0.221734 

GRADES 

53 

0.249394 

HSPR0G2 

54 

-0.073426 

HSPR0G3 

DESCRIPTION 
HIGHEST  LEVEL  MATH  TAKEN  (PRE-ALGEBRA) 
HIGHEST  LEVEL  MATH  TAKEN  (ALGEBRA) 
HIGHEST  LEVEL  MATH  TAKEN  (GEOMETRY) 
HIGHEST  LEVEL  MATH  TAKEN  (ALGEBRA- 2) 
HIGHEST  LEVEL  MATH  TAKEN  (CALCULUS)' 
COMPUTER  CLASS  TAKEN  (YES) 
GRADES  IN  SCHOOL 

HIGH  SCHOOL  PROGRAM  (COLLEGE  PREP) 
HIGH  SCHOOL  PROGRAM  (VOC/TECH) 
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Table  B.16 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1986:    Age  9,  Bridge  A 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-0. 

725251 

OVERALL 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

-0. 

067436 

GENDER2 

SEX  (FEMALE) 

3 

-0. 

011334 

ETHNIC2 

OBSERVED  ETHNICITY  (BLACK) 

4 

0. 

661119 

ETHNIC3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

091786 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

446555 

ST0C2 

SIZE  AND  TYPE  OF  COMJ^UNITY  (HIGH  METRO) 

7 

0. 

186656 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

087980 

REGI0N2 

REGION  (SOUTHEAST) 

9 

0. 

020318 

REGIONS 

REGION  (CENTRAL) 

10 

-0. 

176899 

REGI0N4 

REGION  (WEST) 

11 

0 

434227 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0. 

558842 

PARED3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0, 

608261 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

0 

369200 

PARED 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-0 

900207 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0 

662656 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0 

196782 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

0 

320011 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

0 

.060679 

E2  X  SEX 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

20 

-0 

049282 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

-0 

006737 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

-0 

.430534 

E2  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

-0 

438015 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,   POST  HS) 

24 

-0 

474353 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

25 

-0 

393091 

E2  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

-0 

854606 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

27 

-0 

.772768 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,   POST  HS) 

28 

-0 

.794394 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

-0 

.742478 

E3  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

0 

.220517 

E4  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

31 

0 

.138926 

E4  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

32 

0 

.013411 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 

33 

0 

.414963 

TV. 0-2 

TV  WATCHING  (0-2  HOURS) 

34 

0 

.491512 

TV. 3-5 

TV  WATCHING  (3-5  HOURS) 

35 

0 

.303934 

TV. 6+ 

TV  WATCHING  (6+  HOURS) 

36 

-0 

.322312 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (ALWAYS) 

37 

0 

.044631 

H0MELNG2 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES) 

38 

-0 

.010885 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

39 

-0 

.102755 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

40 

0 

.179859 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

41 

-0 

.083553 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

42 

0 

.109314 

COMPUTER 

ARE  YOU  STUDYING  COMPUTERS?  (YES) 

43 

-0 

.088235 

DRACE2 

DERIVED  RACE  (BLACK) 

44 

-0 

.240777 

DRACE3 

DERIVED  RACE  (HISPANIC) 

45 

0 

.271526 

DRACE4 

DERIVED  RACE  (ASIAN  AMERICAN) 

541 
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Table  B.17 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1986:    Age  9,  Bridge  B 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-1.296606 

OVERALL 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

0.052957 

GENDER2 

SEX  (FEMALE) 

3 

-0.252366 

ETHNIC 2 

OBSERVED  ETHNICITY  (BLACK) 

4 

0.067568 

ETHNIC 3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0.892365 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0.782783 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0.520928 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0.073264 

REGI0N2 

REGION  (SOUTHEAST) 

9 

-0.075212 

REGI0N3 

REGION  (CENTRAL) 

10 

-0. 144994 

REGI0N4 

REGION  (WEST)* 

11 

0. 218180 

PARED 2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0.  526834 

PARED 3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0.475034 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

0. 278631 

PARED_ 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-1. 344181 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0.684191 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0.235841 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

0.266246 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

-0.152639 

E2  X  SEX 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

20 

-0.197325 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

-0.403283 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

-0.074566 

E2  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

-0.140388 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

24 

-0.099987 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

25 

-0.063112 

E2  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

-0.112575 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

27 

-0.580280 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

28 

-0.247772 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

-0.228473 

E3  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

-0.546176 

E4  X  PE4 

ETHNICITY'  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

31 

-0.449937 

E4  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

32 

-0.055567 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 

33 

0.347712 

TV. 0-2 

TV  WATCHING  (0-2  HOURS) 

34 

0.394897 

TV, 3-5 

TV  WATCHING  (3-5  HOURS) 

35 

0.215672 

TV.6+ 

TV  WATCHING  (6+  HOURS) 

36 

-0.232707 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (ALWAYS) 

37 

0.102350 

H0MELNG2 

OTHER  LANGUAGE  AT  HOME  (SOMETIMES) 

38 

0.150033 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

39 

-0.087639 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

40 

0.322346 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

41 

-0.065818 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

42 

0.006116 

COMPUTER 

ARE  YOU  STUDYING  COMPUTERS?  (YES) 

43 

-0.068882 

DRACE2 

DERIVED  RACE  (BIACK) 

44 

-0.165760 

DRACE3 

DERIVED  RACE  (HISPANIC) 

45 

-0.121774 

DRACE4 

DERIVED  RACE  (ASIAN  AMERICAN) 
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Table  B.18 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1986:    Age  13,  Bridge  A 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-0. 

968790 

OVERALL 

OVERALL  CONSTANT  ' 1 '  FOR  EVERYONE 

2 

-0. 

209290 

GENDER2 

SEX  (FEMALE) 

3 

-0. 

452350 

ETHNIC2 

OBSERVED  ETHNICITY  (BLACK) 

4 

-0. 

214310 

ETHNIC3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

000880 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

479050 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0. 

225560 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

065080 

REG I ON 2 

REGION  (SOUTHEAST) 

9 

-0. 

035780 

REGIONS 

REGION  (CENTRAL) 

10 

-0. 

135580 

REGI0N4 

REGION  (WEST) 

11 

-0. 

031800 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0. 

074200 

PARED 3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0. 

114010 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

-0 

162160 

PARED_ 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-0. 

483090 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0. 

316280 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0. 

090490 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

0 

178190 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

0 

081260 

E2  X  SEX 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

20 

0 

002940 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

0 

219890 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

0 

086210 

E2  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

-0 

033540 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,   POST  HS) 

24 

0 

030460 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

25 

0 

251810 

E2  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

-0 

052030 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

27 

0 

130550 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

28 

-0 

097660 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

-0 

038730 

E3  X  PE_ 

ETHNICITY  BY ^PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

0 

.219560 

E4  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD 

31 

0 

175790 

E4  X  PE 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

32 

-0 

.087090 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 

33 

-0 

.523890 

TV. 0-2 

TV  WATCHING  (0-2  HOURS) 

34 

-0 

.502250 

TV . 3  -  5 

TV  WATCHING  (3-5  HOURS) 

35 

-0 

.689270 

TV.  6+ 

TV  WATCHING  (6+  HOURS) 

36 

-0 

.059210 

HW-NO 

HOMEWORK  (NONE  ASSIGNED) 

37 

0 

.115340 

HW-YES 

HOMEWORK  (YES  -  SOME  AMOUNT) 

38 

-0 

.023640 

HW-2345 

HOMEWORK  AMOUNT  (LINEAR) 

39 

-0 

.189330 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (MOSTLY,  ALWAYS) 

40 

0 

.075950 

H0MELNG2 

OTHER  LANGUAGE  AI  HOME  (OCCASIONALLY,   1/2  TIME) 

41 

0 

.028270 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

42 

0 

.061320 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

43 

0 

.159450 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

44 

0 

.136840 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

45 

0 

.359350 

GRADES 

GRADLS  IN  SCHOOL 
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Table  B.18 
(continued) 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

46 

0.455550 

TMATH2 

TYPE  OF 

MATH  CLASS  (REGULAR  MATH) 

47 

0.645330 

TMATH3 

TYPE  OF 

MATH  CLASS  (PRE -ALGEBRA) 

48 

0.898040 

TMATH45 

TYPE  OF 

MATH  CLASS  (ALGEBRA,  OTHER) 

49 

-0.003550 

COMPUTER 

ARE  YOU 

STUDYING  COMPUTERS?  (YES) 

50 

-0.038190 

DRACE2 

DERIVED 

RACE  (BLACK) 

51 

-0.181510 

DRACE3 

DERIVED 

RACE  (HISPANIC) 

52 

0.258680 

DRACE4 

DERIVED 

RACE  (ASIAN  AMERICAN) 
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Table  B.19 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1986:    Age  13,  Bridge  B 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-2. 

314038 

OVERALL 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

-0. 

217852 

GENDER2 

SEX  (FEMALE) 

3 

-0. 

495203 

ETHNIC2 

OBSERVED  ETHNICITY  (BLACK) 

4 

0. 

479798 

ETHNIC 3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

159407 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

272203 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0. 

159015 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

200327 

REG I ON 2 

REGION  (SOUTHEAST) 

9 

-0. 

130221 

REGIONS 

REGION  (CENTRAL) 

10 

-0. 

290191 

REGI0N4 

REGION  (WEST) 

11 

-0. 

026075 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0. 

155467 

PARED 3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0. 

207214 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GEIAD) 

14 

0. 

088771 

PARED 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

-0. 

640996 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

0. 

231757 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0. 

169866 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

0. 

239980 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

0 

130466 

E2  X  SEX 

ETHNICITY  BY  GENDER  (BLACK  FEMALE) 

20 

-0 

101624 

E3  X  SEX 

ETHNICITY  BY  GENDER  (HISPANIC  FEMALE) 

21 

-0 

481779 

E4  X  SEX 

ETHNICITY  BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

0 

062448 

E2  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

23 

0 

.129460 

E2  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

24 

-0 

.101520 

E2  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

25 

0 

.057715 

E2  X  PE 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

26 

-0 

.187339 

E3  X  PE2 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

27 

-0 

.114956 

E3  X  PE3 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

28 

-0 

.346426 

E3  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

29 

-0 

.231446 

E3  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

30 

0 

.436624 

E4  X  PE4 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

31 

0 

.427622 

E4  X  PE_ 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

32 

0 

.163995 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 

33 

0 

.885902 

TV. 0-2 

TV  WATCHING  (0-2  HOURS) 

34 

0 

.834638 

TV. 3-5 

TV  WATCHING  (3-5  HOURS) 

35 

0 

.734312 

TV.6+ 

TV  WATCHING  (6+  HOURS) 

36 

0 

.036860 

HW-NO 

HOMEWORK  (NONE  ASSIGNED) 

37 

0 

.131824 

HW-YES 

HOMEWORK  (YES  -  SOME  AMOUNT) 

38 

-0 

.001523 

HW-2345 

HOMEWORK  AMOUNT  (LINEAR) 

39 

-0 

.327862 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (MOSTLY,  ALWAYS) 

40 

0 

.037094 

H0MELNG2 

OTHER  LANGUAGE  AT  HOME  (OCCASIONALLY,   1/2  TIME) 

41 

0 

.331032 

HL2  X  E2 

HOME  LANG  BY  ETHNICITY  (OFTEN,  BLACK) 

42 

0 

.075593 

HLl  X  E2 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  BLACK) 

43 

0 

.246166 

HL2  X  E3 

HOME  LANG  BY  ETHNICITY  (OFTEN,  HISPANIC) 

44 

0 

.051226 

HLl  X  E3 

HOME  LANG  BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

45 

0 

.380695 

GRADES 

GRADES  IN  SCHOOL 
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Table  B.19 
(continued) 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

46 

0.299062 

TMATH2 

TYPE  OF 

MATH  CLASS  (REGULAR  MATH) 

47 

0.530130 

TMATH3 

TYPE  OF 

MATH  CLASS  (PRE-ALGEBRA) 

48 

0.373462 

TMATH45 

TYPE  OF 

MATH  CLASS  (ALGEBRA,  OTHER) 

49 

0.038474 

COMPUTER 

ARE  YOU 

STUDYING  COMPUTERS?  (YES) 

50 

-0.195711 

DRACE2 

DERIVED 

RACE  (BLACK) 

51 

-0.416190 

DRACE3 

DERIVED 

RACE  (HISPANIC) 

52 

-0.144319 

DRACE4 

DERIVED 

RACE  (ASIAN  AMERICAN) 
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Table  B.20 


Estimated  Effects  for  Mathematics  Trend  Conditioning  Variables 

1986:    Age  17 


ESTIMATED 

VARIABLE 

EFFECT 

LABEL 

DESCRIPTION 

1 

-2. 

485863 

OVERALL 

OVERALL  CONSTANT  '1'  FOR 

EVERYONE 

2 

-0. 

203894 

GENDER2 

SEX  (FEMALE) 

3 

0. 

124491 

ETHNIC2 

OBSERVED  ETHNICITY  (BLACK) 

4 

-0. 

431167 

ETHNIC3 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

0. 

212997 

ETHNIC4 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

0. 

413316 

ST0C2 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

0. 

352628 

ST0C3 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

-0. 

005895 

REGI0N2 

REGION  (SOUTHEAST) 

9 

0. 

019766 

REGI0N3 

REGION  (CENTRAL) 

10 

-0. 

045038 

REGI0N4 

REGION  (WEST) 

11 

0. 

030552 

PARED2 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

0. 

101183 

PARED 3 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

0. 

121523 

PARED4 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

-0. 

060429 

PARED_ 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

-0 

205362 

<  MODALG 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

-0 

061003 

>  MODALG 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

0 

018149 

H0MEITM3 

ARTICLES  IN  HOME  (YES  TO 

3) 

18 

0 

053439 

H0MEITM4 

ARTICLES  IN  HOME  (YES  TO 

4) 

19 

0 

053462 

E2 

X  SEX 

ETHNICITY 

BY  GENDER  (BLACK  FEMALE) 

20 

0 

096039 

E3 

X  SEX 

ETHNICITY 

BY  GENDER  (HISPANIC  FEMALE) 

21 

-0 

172182 

E4 

X  SEX 

ETHNICITY 

BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

22 

-0 

045511 

E2 

X  PE2 

ETHNICITY 

BY  PARENT'S  ED 

(BLACK,  HS  GRAD) 

23 

-0 

.019722 

E2 

X  PE3 

ETHNICITY 

BY  PARENT'S  ED 

(BLACK,  POST  HS) 

24 

-0 

025977 

E2 

X  PE4 

ETHNICITY 

BY  PARENT'S  ED 

(BLACK,  COLLEGE  GRAD) 

25 

-0 

026264 

E2 

X  PE 

ETHNICITY 

BY  PARENT'S  ED 

(BLACK,  UNKNOWN) 

26 

0 

.241818 

E3 

X  PE2 

ETHNICITY 

BY  PARENT'S  ED 

(HISPANIC,  HS  GRAD) 

27 

0 

.249675 

E3 

X  PE3 

ETHNICITY 

BY  PARENT'S  ED 

(HISPANIC,   POST  HS) 

28 

0 

.296959 

E3 

X  PE4 

ETHNICITY 

BY  PARENT'S  ED 

(HISPANIC,  COLLEGE) 

29 

0 

.009964 

E3 

X  PE 

ETHNICITY 

BY  PARENT'S  ED 

(HISPANIC,  UNKNOWN) 

30 

-0 

.231579 

E4 

X  PE2 

ETHNICITY 

BY  PARENT'S  ED 

(ASIAN  AMER,  HS  GRAD) 

31 

0 

.203965 

E4 

X  PE3 

ETHNICITY 

BY  PARENT'S  ED 

(ASIAN  AMER,  POST  HS) 

32 

0 

.062122 

E4 

X  PE4 

ETHNICITY 

BY  PARENT'S  ED 

(ASIAN  AMER,  COLL  GRAD 

33 

-0 

.090388 

SCH.PRIV 

SCHOOL  TYPE  (PRIVATE) 

34 

1 

.027634 

TV 

.0-2 

TV  WATCHING  (0-2  HOURS) 

35 

0 

.972085 

TV 

.3-5 

TV  WATCHING  (3-5  HOURS) 

36 

0 

.815760 

TV 

.6+ 

TV  WATCHING  (6+  HOURS) 

37 

-0 

.088868 

HW 

-NO 

HOMEWORK 

(NONE  ASSIGNED) 

38 

0 

.258237 

HW 

-YES 

HOMEWORK 

(YES  -  SOME  AMOUNT) 

39 

-0 

.079234 

HW 

-2345 

HOMEWORK  AMOUNT  (LINEAR) 

40 

-0 

.077022 

HOMELNGl 

OTHER  LANGUAGE  AT  HOME  (MOSTLY,  ALWAYS) 

41 

0 

.028441 

H0MELNG2 

OTHER  LANGUAGE  AT  HOME  (OCCASIONALLY,   1/2  TIME) 

42 

0 

.063506 

HL2  X  E2 

HOME  LANG 

BY  ETHNICITY  (OFTEN,  BLACK) 

43 

0 

.034244 

HLl  X  E2 

HOME  LANG 

BY  ETHNICITY  (SOMETIMES,  BLACK) 

44 

0 

.399629 

HL2  X  E3 

HOME  LANG 

BY  ETHNICITY  (OFTEN,  HISPANIC) 

45 

0 

.294080 

HLl  X  E3 

HOME  LANG 

BY  ETHNICITY  (SOMETIMES,  HISPANIC) 

-495- 
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Table  B.20 
(continued) 


ESTIMATED 

X7AD  T  A  BT  I? 

EFFECT 

T  A  TII7T 

46 

INnAirii 

UTrWTTQT   T  T7\7T7T    MATH   TAK"T?Kf    ^  PRF  -  AT  GFRRA'i 

47 

0. 

097243 

NMAinZ 

UTPUT70T   T  T7\7??T    MATH   TAl^FN  ^AT/5FRRA^ 

48 

0. 

324996 

HTPHF^T  T  FVFT    MATH  TAKFN   f  GEOMETRY^) 

49 

0, 

689081 

NMATH4 

HIGHEST  LEVEL  MATH  TAKEN  (ALGEBRA- 2) 

50 

1 

223782 

NMATH5 

HIGHEST  LEVEL  MATH  TAKEN  (CALCULUS) 

51 

0 

.020003 

COMPUTER 

COMPUTER  CLASS  TAKEN  (YES) 

52 

0 

.292099 

GRADES 

GRADES  IN  SCHOOL 

53 

0 

.226475 

HSPR0G2 

HIGH  SCHOOL  PROGRAM  (COLLEGE  PREP) 

54 

-0 

.056057 

HSPR0G3 

HIGH  SCHOOL  PROGRAM  (VOC/TECH) 

55 

-0 

.531790 

DRACE2 

DERIVED  RACE  (BLACK) 

56 

-0 

.345698 

DRACE3 

DERIVED  RACE  (HISPANIC) 

57 

-0 

.125825 

DRACE4 

DERIVED  RACE  (ASIAN  AMERICAN) 

-496- 
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Table  B.21 


Science  Conditioning  Variables,  Grade  3/Age  9 


VARIABLE 

LABEL 

DESCRIPTION 

1 

OVERALL 

1 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

3 

ETHNICITY 

(BLACK) 

4 

ETHNIC3 

4 

ETHNICITY 

(HISPANIC) 

5 

ETHNIC4 

5 

ETHNICITY 

(ASIAN  AMERICAN) 

6 

ST0C2 

6 

7 

ST0C3 

7 

8 

REGI0N2 

8 

REG  I  ON  C  S  OUTHEAS  T 

9 

REGI0N3 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

10 

REGION  (WEST) 

11 

PARED 2 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

ITEMS2 

15 

ITEMS  IN  HOME  (FOUR  OF  THE  FIVE) 

16 

ITEMS3 

16 

ITEMS  IN  HOME  (FIVE  OF  THE  FIVE) 

17 

TV 

17 

HOURS  TV  WATCHING  (LINEAR) 

18 

TV**2 

18 

HOURS  TV  WATCHING  (QUADRATIC) 

19 

HW- 

•YES 

19 

HOMEWORK  (YES  -  SOME  AMOUNT) 

20 

HW- 

'2345 

20 

HOMEWORK  AMOUNT  (LINEAR) 

21 

LM 

BY  E3 

21 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISi  .^IC) 

22 

LM 

BY  E4 

22 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  ASIAN  AMER) 

23 

LM 

BY  E 

23 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  OTHER  ETH) 

24 

LUNCH% 

24 

PERCENT  IN  LUNCH  PROGRAM 

25 

LUNCH_ 

25 

LUNCH  PROGRAM  (MISSING) 

26 

%WHITE49 

26 

PERCENT  WHITE  IN  SCHOOL  (0-49%  WHITE  MINORITY) 

27 

XWHITE79 

27 

PERCENT  WHITE  IN  SCHOOL  (50-79%  INTEGRATED) 

ZWHITEOO 

28 

PERCENT  WHITE  IN  SCHOOL  (80-100%  PREDOMINANTLY) 

28 

E2 

X  SEX 

29 

ETHNICITY 

BY  GENDER  (BLACK  FEMALE) 

29 

E3 

X  SEX 

30 

ETHNICITY 

BY  GENDER  (HISPANIC  FEMALE) 

30 

E4 

X  SEX 

31 

ETHNICITY 

BY  GENDER  (ASIAN  AMERICAN  FEMALE) 

31 

E2 

X  PE2 

32 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  HS  GRAD) 

32 

E2 

X  PE3 

33 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  POST  HS) 

33 

E2 

X  PE4 

34 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  COLLEGE  GRAD) 

34 

E2 

X  PE_ 

35 

ETHNICITY 

BY  PARENT'S  ED  (BLACK,  UNKNOWN) 

35 

E3 

X  PE2 

36 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

36 

E3 

X  PE3 

37 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,   POST  HS) 

37 

E3 

X  PE4 

38 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

38 

E3 

X  PE 

39 

ETHNICITY 

BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

E4 

X  PE2 

40 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

39 

E4 

X  PE3 

41 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

40 

E4 

X  PE4 

42 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

41 

E4 

X  PE_ 

43 

ETHNICITY 

BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 

42 

MA,<MG 

44 

MODAL  AGE 

,  LESS  THAN  MODAL  GRADE 

-497-        r;  o 
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Table  B.21 
(continued) 


43 

44 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 


VARIABLE 
LABEL 

MA,MG 

MA,>MG 

>MA,MG 

SCH  TYPE 

ASK  SW? 

PRESCHl 

#PARENT1 

MOTHER 

MOWORK 

SCI123 

SCI45 

COMPUTER 

SUPERVIS 

MATH  Ql 

SCI  Ql 


DESCRIPTION 

45  MODAL  AGE,  MODAL  GRADE,  MISSING 

46  MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

47  GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

48  SCHOOL  TYPE  (NOT  PUBLIC) 

49  FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

50  WENT  TO  PRESCHOOL  (YES) 

51  SINGLE/MULTIPLE  PARENT  HOME  (MOTHER , FATHER  HOME) 

52  MOTHER  AT  HOME 

53  MOTHER  WORKS  OUTSIDE  (YES) 

54  SPENT  AT  LEAST  ONCE  A  WEEK  STUDYING  SCIENCE 

55  SPENT  <  ONCE  A  WEEK  OR  NEVER  STUDYING  SCIENCE 

56  USE  COMPUTERS  FOR  MATH,  READING,   ETC,  (YES) 

57  ADULT  SUPERVISION  OF  STUDENT  AFTER  SCHOOL(YES) 

58  MATH    QUANTILE  (LINEAR  -1,0,1) 

59  SCIENCE    QUANTILE  (LINEAR  -1,0,1) 


ERLC 
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Table  B,22 


Estimated  Effects  for  C -ience  Conditioning  Variables,  Grade  3/Age  9 


SUBSCALE    SUBSCALE  SUBSCALE 


1 

3 

1 

-1 

.027950 

-1 

.539603 

-0 

41 1 86S 

2 

-0 

.036995 

0 

.034485 

-0 

yj  m 

^  <L*-r     1.  ^ 

3 

0 

.004488 

-0 

.  220916 

yj  . 

7  O  7  1.  o 

4 

-0 

.046322 

-0 

yj  % 

5 

-0 

.  512803 

0 

.      ^  J  J  O  1. 

0 
\j , 

fi'^Ql  Ql 

6 

0 

. 317467 

0 

1 6S71 6 

.  X  V/  ^  /  X 

n 

^  D  L  1  Jyj 

7 

0 

. 195622 

0 

106610 

n 

yj  , 

1  7'^Qft9 

8 

-0 

. 051878 

0 

.  049463 

n 
yj  , 

yj  c.  Dyjyj'-r 

9 

-0 

058628 

0 

. 014442 

u . 

yj D  I  *+o ^ 

10 

-0 

. 099356 

0 

0067S1 

n 

c\fy^  HAS 

11 

0 

.  209816 

0 

.  033078 

n 

yj , 

909  'XL.O 

12 

0 

.418804 

0 

.  ^  X  1.  ^  ^ 

n 
yj , 

J  ^*40  O  O 

13 

0 

. 371804 

0 

.  320744 

n 

yj  m 

14 

0 

.228783 

0 

.271383 

0 . 

1  '^S7SQ 
/ ^y 

15 

0 

.052308 

0 

.  070849 

n 

yj  • 

111  '^nQ 

16 

0 

.081124 

0 

.228607 

0 

1  7009^. 
X  /  yjyj c.'-r 

17 

0 

.078107 

0 

.235255 

0, 

216679 

18 

-0 

.014847 

-0 

.032669 

-0. 

034084 

19 

0 

.078969 

-0 

,076431 

0 . 

112195 

20 

-0 

.001100 

-0 

,027368 

-0 . 

021323 

21 

0 

.090482 

0 

.152099 

0, 

185703 

22 

0 

,099780 

0 

.127217 

0. 

040464 

23 

-0 

.133542 

-0 

.152438 

-0, 

230931 

24 

-0 

.259334 

-0 

.166585 

-0. 

382387 

25 

-0 

.137761 

-0 

.072979 

-0. 

158428 

26 

-0 

.141557 

-0 

.134732 

-0, 

198346 

27 

-0 

.028554 

-0 

.066428 

-0. 

108059 

28 

0 

.092295 

0 

.008121 

0. 

106356 

29 

0 

.107580 

0 

.099879 

-0. 

055230 

30 

0 

.312742 

-0 

,301925 

-0, 

162949 

31 

-0 

.336560 

-0 

.108481 

-0. 

399386 

32 

-0 

,479056 

-0 

.371290 

-0. 

380324 

33 

-0 

,345508 

-0 

,209929 

-0. 

205010 

34 

-0 

.311267 

-0 

.236645 

-0, 

136088 

35 

-0 

.174188 

-0 

.054116 

-0, 

390267 

36 

-0 

.255924 

-0 

.261684 

-0, 

282314 

37 

-0 

.287467 

-0 

.240709 

-0, 

287834 

38 

-0 

,226640 

-0 

,167789 

-0, 

100105 

39 

0 

,383161 

«1 

,628189 

-1, 

164638 

40 

0 

,150046 

-1 

.295492 

-1 

460198 

41 

0 

,440562 

-0 

,886048 

-0 

438206 

42 

0 

,050570 

-0 

,759013 

-0 

815707 

43 

-0 

.468759 

-0 

,652030 

-0 

636868 

44 

0 

.105295 

-0 

.003158 

0 

035390 

ERIC 
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Table  B.22 
(continued) 


SUBSCALE    SUBSCALE  SUBSCALE 


1 

i 

6 

45 

0  . 

11/.  CCiO 

n 
u . 

0 . 

272590 

46 

-0 . 

1  ^ftOI  7 

-0 . 

225379 

/i  7 

«n 
"  \j , 

070884 

-0. 

131958 

-0. 

151235 

48 

-0. 

034763 

-0. 

005629 

-0. 

070424 

49 

0. 

144045 

0. 

082908 

0. 

120839 

50 

0 

147113 

0. 

137197 

0. 

067721 

51 

0 

052701 

0 

059180 

0 

198927 

52 

-0 

.038439 

0 

.029265 

0 

.002207 

53 

-0 

.258651 

0 

.263742 

-0 

.051063 

54 

-0 

.385471 

0 

.166218 

-0 

.213505 

55 

0 

.033335 

0 

.039578 

0 

,043706 

56 

0 

.065488 

0 

.118883 

0 

.100908 

57 

-0 

.280087 

-0 

.249134 

-0 

.327255 
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Table  B.23 


Science  Conditioning  Variables,  Grade  7/Age  13 


VARIABLE 

LABEL 

DESCRIPTION 

1 

OVERALL 

1 

X 

2 

GENDER2 

2 

<?FY  ^FFMATF^ 

3 

ETHNIC2 

FTHNTCITY  ^RTACK"^ 

4 

ETHNIC3 

4 

ETHNICITY  ^HI^PANIC^ 

I-i  ±L  iVt  XV^XxX     yllXOx  cWl  X  / 

5 

ETHNTC4 

5 

ETHNICITY  ^A<;iAN  AMFRICAN^ 
x>  X 1  ll%  X  \^  X  X  X    y  fio  xruX   ni  iiiiis.  x  v»jnxN  j 

6 

ST0C2 

V 

^I7F  AND  TYPF  OF  nOMMTTNITY   /'HTPH  M^TD^^ 

7 

ST0C3 

7 

^I7F  AND  TYPF  OF  rOMMlTNTTY   ^TJHT  UTPW   HD    T  ^TJ^ 

8 

Q 

PFflTON  ^<;nTTTHFA<;T^ 

9 

REGIONS 

XVi_i  \J  X  v^x^  ^ 

Q 
✓ 

RFC  ION  ^CFNTRAT 

10 

RFGI0N4 

1  n 

RFfJTON  ^UF<;T^ 

£\.Iltv7  XOii  ^WHtOX^ 

11 

PARED  2 

1  1 

PARFNT^    FDTTrATTON    ^HTPH   CPWnnT  PRAD^ 

12 

X  flx\.Iltl^  ^ 

13 

PARED4 

13 

PARENTS   FDITGATION   ^rOTIFflF  PRAfi^ 

14 

PARED 

14 

PARFNT^   FDITGATION    ^MT^^TMP      T   nnW'T  ^^'MnU^ 

15 

ITEMS2 

15 

ITEMS   IN  HOME   ^FOTJR  OF  THF  FIW^ 

16 

ITEMS 3 

16 

17 

TV 

17 

HOURS  TV  WATCHING  niNFAR'i 

18 

TV**2 

18 

HOURS  TV  WATCHING  (QUADRATIC) 

19 

HW-NO 

19 

HOMEWORK  (DON'T  HAVE  ANY) 

20 

HW-YES 

20 

HOMEWORK  (YES  -  SOME  AMOUNT) 

21 

HW-2345 

21 

HOMEWORK  AMOUNT  (LINEAR) 

22 

m  BY  E3 

22 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISPANIC) 

23 

LM  BY  E4 

23 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  ASIAN  AMER) 

24 

LM  BY  E_ 

24 

LANGUAGE  MINORITY  BY  ETHNICITY  ^YES    OTHER  ETH") 

25 

LUNCH% 

25 

PERCENT  IN  LUNCH  PROGRAM 

26 

LUNCH 

26 

LUNCH  PROGRA^^  CMISSINO 

27 

%WHITE49 

27 

PERCENT  WHITE  IN  SCHOOL  ^'0-49%  WHITE  MINORITVi 

28 

%WHITE79 

28 

PERCENT  WHITE  IN  SCHOOL     '^Di^  -  795!: 

X           L^uy  X     n  XX  X  X  xj    X  XI    kj  WX  iw  w  x^    V'^v.'^     i  y  ro  j 

XWHITEOO 

29 

PERCENT  WHITE  IN  SCHOOT    ^80-1005!!  PRFDOMINANTT 

X  X-«X\.V^X-«X^  X      WXIXXX^     XXl      \J\JL\\J\JL^      \\J\J      l.\J\J  f9      X  XVlltJL^L^i  iXJ.^Xxl/(  X  X<  X  J 

29 

E2  X  SEX 

30 

ETHNICITY  BY  GENDER  CBLACK  FEMAT  F'i 

X^  X  X  XXI  Xp  \^  Xp  X  A       X^  X            X^X^  J>^  X^XX       I  J_i*  j|_-tTA  v-*  x\_      X   x-aX  Xxxx^X-tf  J 

30 

E3  X  SEX 

31 

ETHNICITY  BY  GENDER  CHISPANIC  FEMALE") 

X^  ^  X          ^  Xi'  ^  ^  A       X^  X       X^X^X^  X/  X^XX       \  X  1  Xp       X  XXX^  Xp  \^       X    X^x  JxxX^X^  J 

31 

E4  X  SEX 

32 

ETHNICITY  BY  GENDER  CASIAN  AMERICAN  FFMATF'i 

X          X  Vi' X  X  X     JJ  X     xJXjXIX/X^Xn.      ^  tilJ  X*XX1     rVL  X  XIjXx  X  V^aU  1  XXjXXr\X^Xjl 

32 

E2  X  PE2 

33 

ETHNICITY  BY  PARENT'S  ED  ('RTJVCK     HS  GRAD'i 

X^  X  X  £X^  X  Vi'  XXX       X^  X       X  aXXxXpJXI  X       lJ       1^1^       V        jt_-fTfL  V*  JCV          X  Ilj      x^xxxxJ-/  1 

33 

E2  X  PE3 

34 

ETHNICITY  BY  PARENT'S  ED  (BLACK,  POST  HS) 

34 

E2  X  PE4 

35 

ETHNICITY  BY  PARENT'S  ED  (BLACK i  COLLEGE  GRAD) 

35 

E2  X  PE_ 

36 

ETHNICITY  BY  PARENT'S  ED  CBLACK  UNKNOWN") 

36 

E3  X  PE2 

37 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  HS  GRAD) 

37 

E3  X  PE3 

38 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  POST  HS) 

38 

E3  X  PE4 

39 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  COLLEGE) 

39 

E3  X  PE_ 

40 

ETHNICITY  BY  PARENT'S  ED  (HISPANIC,  UNKNOWN) 

40 

E4  X  PE2 

41 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

41 

E4  X  PE3 

42 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  POST  HS) 

42 

E4  X  PE4 

43 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

43 

E4  X  PE_ 

44 

ETHNICITY  BY  PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 
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Table  B.23 
(continued) 


VARIABLE 

LABEL 

44 

MA,<MG 

45 

MA,MG 

46 

MA,>MG 

47 

>MA,MG 

48 

SCH  TYPE 

49 

ASK  SW? 

50 

PRESCHl 

51 

#PARENT1 

52 

MOTHER 

53 

MOWORK 

54 

GOMPUTI;R 

55 

MATH2 

56 

MATH3 

57 

MATH45 

58 

SCIENCE2 

59 

SCIENCE3 

60 

SCIENCE4 

61 

SCIENCES 

62 

GRADES 

63 

MATH  Ql 

64 

SCI  Ql 

DESCRIPTION 

45  MODAL  AGE,  LESS  THAM  MODAL  GRADE 

46  MODAL  AGE,  MODAL  GRADE,  MISSING 

47  MODAL  AGE,  GREATER  THAN  MODAL  GRADE 

48  GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

49  SCHOOL  TYPE  (NOT  PUBLIC) 

50  FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

51  WENT  TO  PRESCHOOL  (YES) 

52  SINGLE/MULTIPLE  PARENT  HOME  (MOTHER , FATHER  HOME) 

53  MOTHER  AT  HOME 

5A  MOTHER  WORKS  OUTSIDE  (YES) 

55  USE  COMPUTERS  FOR  MATH,  READING,  ETC.  (YES) 

56  TYPE  OF  MATH  CLASS  (REGULAR  MATH) 

57  TYPE  OF  MATH  CLASS (PRE -ALGEBRA) 

58  TYPE  OF  MATH  CLASS (ALGEBRA) 

59  STUDYING  IN  SCIENCE  THIS  YEAR  (LIFE  SCIENCE) 

60  STUDYING  IN  SCIENCE  THIS  YEAR  (PHYSICAL  SCIENCE) 

61  STUDYING  IN  SCIENCE  THIS  YEAR  (EARTH  SCIENCE) 

62  STUDYING  IN  SCIENCE  THIS  YEAR  (GENERAL  SCIENCE) 

63  GRADES  IN  SCHOOL  (LINEAR) 

64  MATH    QUANTILE  (LINEAR  -1,0,1) 

65  SCIENCE    QUANTILE  (LINEAR  -1,0,1) 
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Table  B.24 


Estimated  Effects  for  Science  Conditioning  Variables,  Grade  7/Age  13 


SUBSCALE    SUBSCALE    SUBSCALE    SUBSCALE  SUBSCALE 


1 

2 

3 

4 

5 

1 

-0. 

921900 

-0. 

728416 

-0. 

964766 

-0. 

403002 

045989 

2 

-0. 

188817 

-0. 

289175 

-0 . 

024590 

-0 . 

526984 

99fiS94 

3 

-0. 

344867 

-0. 

609363 

-0 . 

174167 

-0 

yj  • 

396886 

O  /  \J\J  ^*-r 

4 

-0. 

278339 

-0. 

327569 

-0. 

227757 

-0. 

063358 

^  ^  ^  J  t  \J 

5 

-1. 

058032 

-1, 

822311 

0. 

789686 

-1. 

016836 

0 

J  J  V/ ^'t 

6 

0, 

160177 

0. 

145207 

0. 

004400 

V/  w             %/  V/ 

0. 

087345 

0. 

145490 

7 

0, 

095899 

0, 

068574 

0, 

013040 

0. 

118676 

0, 

100085 

8 

0. 

029778 

-0. 

017066 

-0. 

015193 

-0. 

049297 

0. 

019044 

9 

0, 

057041 

0. 

026123 

0. 

062015 

0. 

041668 

0 . 

10 

-0, 

033711 

-0 

067379 

-0. 

032482 

-0. 

0, 

011705 

11 

0. 

029428 

-0. 

054719 

0. 

093961 

0. 

102967 

0. 

071009 

12 

0, 

176754 

0 

151999 

0. 

197954 

0, 

223748 

0. 

241014 

13 

0 

194384 

0 

101753 

0 

232376 

0, 

214939 

0 

296519 

14 

-0 

044960 

-0, 

063955 

0 

086042 

0, 

039230 

0 

059112 

15 

0, 

085018 

0, 

038272 

0 

063404 

0. 

153887 

0, 

121538 

16 

0 

094808 

0 

104775 

0 

095784 

0, 

156526 

0, 

161213 

17 

0, 

080265 

0 

040780 

0 

144271 

0, 

053809 

0, 

087588 

18 

-0 

013240 

-0 

006388 

-0, 

021666 

-0, 

010680 

-0, 

017068 

19 

0 

036270 

-0 

210399 

-0 

041576 

-0, 

490439 

-0 

078815 

20 

0, 

100315 

-0 

227302 

0 

053470 

-0, 

428770 

-0. 

170859 

21 

0 

007153 

0 

013630 

0 

026170 

-0 

018446 

0 

020861 

22 

-0 

125860 

0 

019754 

0 

025339 

-0 

043756 

-0 

023216 

23 

-0 

225117 

-0 

352867 

-0 

321131 

-0 

125152 

-0 

625920 

24 

0 

035864 

-0 

040543 

-0 

069561 

0 

125382 

0 

002168 

25 

-0 

.029924 

-0 

206254 

-0 

059946 

0 

024705 

-0 

032774 

26 

-0 

.029282 

-0 

.183671 

-0 

066899 

-0 

008394 

-0 

.030263 

27 

-0 

.176457 

-0 

144600 

-0 

113581 

-0 

244936 

-0 

240365 

28 

-0 

.031899 

-0 

.064580 

-0 

.013255 

-0 

036387 

-0 

.040290 

29 

0 

.034209 

0 

.121098 

-0 

060865 

-0 

013686 

0 

30 

0 

.075908 

0 

.176640 

-0 

.013788 

0 

.027655 

0 

.102128 

31 

0 

.202159 

-0 

.175074 

-0 

.530370 

-0 

.266771 

-0 

.102259 

32 

0 

.037020 

0 

.134764 

-0 

.139545 

-0 

.119350 

0 

.488599 

33 

0 

.125354 

0 

.062806 

-0 

.026240 

0 

.079002 

0 

.788805 

34 

-0 

.028683 

0 

.133492 

-0 

.116936 

-0 

.137285 

0 

.385568 

35 

0 

.022668 

0 

.019560 

-0 

.181972 

-0 

.094152 

0 

.491215 

36 

0 

.096676 

-0 

.280242 

-0 

.026605 

0 

.033987 

0 

.088311 

37 

0 

.140235 

-0 

.005111 

0 

.133751 

-0 

.068758 

-0 

.1169^7 

38 

-0 

.071345 

-0 

.151671 

-0 

.057318 

-0 

.088970 

-0 

.1981;.:? 

39 

0 

.216444 

-0 

.047763 

-0 

.031577 

-0 

.022252 

-0 

.032317 

40 

1 

.187678 

2 

.276199 

-0 

.795834 

0 

.792370 

-0 

.362707 

41 

0 

.90r998 

1 

.792808 

-0 

.384851 

1 

.839425 

-0 

.510366 

42 

1 

.264378 

2 

.124688 

-0 

.732842 

1 

.537907 

0 

.112796 

43 

1 

.105844 

2 

.423252 

-0 

.545686 

1 

.579802 

-0 

.025581 

44 

-0 

.406121 

-0 

.436965 

-0 

.429181 

-0 

.448541 

-0 

.552704 
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Table  B.24 
(continued) 


SUBSCALE 

SUBSCALE 

SUBSCALE 

SUBSCALE 

SUBSCALE 

1 

i. 

2 

3 

4 

5 

~  u . 

\J  m 

054944 

-0. 

070840 

-0. 

090188 

-0. 

137412 

u . 

n 

088229 

0. 

099190 

0. 

352348 

0. 

174274 

"  u . 

i.O  i. ^ 

265747 

-0. 

270989 

-0. 

232510 

-0. 

313463 

Aft 

-u . 

n 

0 . 

012268 

-0. 

071327 

0. 

017841 

AO 

-  u , 

RA9 
U3  z  o»+z 

081 7A6 

-0 . 

063706 

-0. 

085566 

0. 

011082 

n 
u . 

nAQQ9 1 

n 

048095 

0 . 

055173 

0. 

053287 

0. 

017543 

D  L 

u . 

UOU J  J  u 

✓J 

0. 

008395 

-0. 

021667 

0. 

066283 

3  Z 

u . 

UO 10 1 J 

-  n 

nfii  Q'^8 

U  O  J.  7  J  v> 

-0 , 

013836 

-0 . 

009469 

0. 

011801 

-  u . 

09  A1 OA 

•  u 

m  9SS0 

-0 

001362 

-0 

070962 

-0 

066507 

c;a 

-0 

101173 

-0 

074390 

-0 

110376 

-0 

066353 

-0 

089433 

55 

-0 

.130518 

0 

.083639 

0 

.017161 

0 

.178218 

0 

.101881 

56 

0 

.008173 

0 

.276039 

0 

.229248 

0 

.326113 

0 

.290042 

57 

-0 

.102004 

0 

.236990 

0 

.001423 

0 

.264171 

0 

.162667 

58 

0 

.256450 

0 

.224210 

0 

.171521 

0 

.161651 

0 

.197056 

59 

0 

.091923 

0 

.214919 

0 

.206680 

0 

.139448 

0 

.144136 

60 

0 

0 

.184529 

0 

.147927 

0 

.139468 

0 

.091654 

61 

0 

.269343 

0 

.304336 

0 

.237593 

0 

.247061 

0 

.288282 

62 

0 

.252771 

0 

.231289 

0 

.224300 

0 

.186930 

0 

.285746 

63 

-0 

.260643 

-0 

.212966 

-0 

.291113 

-0 

.294428 

-0 

.280580 

64 

-0 

.135874 

-0 

.179473 

-0 

.186960 

-0 

.060628 

-0 

.128474 

ERIC 
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Table  B.25 


Science  Conditioning  Variables,  Grade  11/Age  17 


VARIABLE 

LABEL 

DESCRIPTION 

1 

OVERALL 

1 

OVERALL  CONSTANT  '1'   FOR  EVERYONE 

2 

GENDER2 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

3 

ETHNICITY 

(BLACK) 

4 

ETHNIC 3 

4 

ETHNICITY 

(HISPANIC) 

5 

ETHNIC4 

5 

ETHNICITY 

(ASIAN  AMERICAN) 

6 

ST0C2 

6 

SIZE  AND  TYPE 

OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

7 

SIZE  AND  TYPE 

OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

8 

REGION  (SOUTHEAST) 

9 

REGI0N3 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

10 

REGION  (WEST) 

11 

PARED 2 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

ITEMS2 

15 

ITEMS  IN  HOME 

(FOUR  OF  THE  FIVE) 

16 

ITEMS3 

16 

ITEMS  IN  HOME 

(FIVE  OF  THE  FIVE) 

17 

TV 

17 

HOURS  TV  WATCHING  (LINEAR) 

18 

XV**  2 

18 

HOURS  TV  WATCHING  (QUADRATIC) 

19 

HW- 

NO 

19 

HOMEWORK  (DON'T  HAVE  ANY) 

20 

HW- 

YES 

20 

HOMEWORK  (YES 

-  SOME  AMOUNT) 

21 

HW- 

2345 

21 

HOMEWORK  AMOUNT  (LINEAR) 

22 

LM 

BY  E3 

22 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  HISPANIC) 

23 

LM 

BY  E4 

23 

LAI^GUAGE  MINORITY  BY  ETHNICITY  (YES,  ASIAN  AMER) 

24 

LM 

BY  E_ 

24 

LANGUAGE  MINORITY  BY  ETHNICITY  (YES,  OTHER  ETH) 

25 

LUNCH% 

25 

PERCENT  IN  LUNCH  PROGRAM 

26 

LUNCH_ 

26 

LUNCH  PROGRAM 

(MISSING) 

27 

%WHITE49 

27 

PERCENT  WHITE 

IN  SCHOOL  (0-49%  WHITE  MINORITY) 

28 

%WHITE79 

28 

PERCENT  WHITE 

IN  SCHOOL  (50-79%  INTEGRATED) 

XWHITEOO 

29 

PERCENT  WHITE 

IN  SCHOOL  (80-100%  PREDOMINANTLY) 

29 

E2 

X  SEX 

30 

ETHNICITY 

BY 

GENDER  (BLACK  FEMALE) 

30 

E3 

X  SEX 

31 

ETHNICITY 

BY 

GENDER  (HISPANIC  FEMALE) 

31 

E4 

X  SEX 

32 

ETHNICITY 

BY 

GENDER  (ASIAN  AMERICAN  FEMALE) 

32 

E2 

X  PE2 

33 

ETHNICITY 

BY 

PARENT'S  ED  (BLACK,  HS  GRAD) 

33 

E2 

X  PE3 

34 

ETHNICITY 

BY 

PARENT'S  ED  (BLACK,   POST  HS) 

34 

E2 

X  PE4 

35 

ETHNICITY 

BY 

?^^JIENT'S  ED  (BLACK,  COLLEGE  GRAD) 

35 

E2 

X  PE_ 

36 

ETHNICITY 

BY 

PARENT'S  ED  (BLACK,  UNKNOWN) 

36 

E3 

X  PE2 

37 

ETHNICITY 

BY 

PARENT'S  ED  (HISPANIC,  HS  GRAD) 

37 

E3 

X  PE3 

38 

ETHNICITY 

BY 

PARENT'S  ED  (HISPANIC,   POST  HS) 

38 

E3 

X  PE4 

39 

ETHNICITY 

BY 

PARENT'S  ED  (HISPANIC,  COLLEGE) 

39 

E3 

X  PE_ 

40 

ETHNICITY 

BY 

PARENT'S  ED  (HISPANIC,  UNKNOWN) 

40 

E4 

X  PE2 

41 

ETHNICITY 

BY 

PARENT'S  ED  (ASIAN  AMER,  HS  GRAD) 

41 

E4 

X  PE3 

42 

ETHNICITY 

BY 

PARENT'S  ED  (ASIAN  AMER.   POST  HS) 

42 

E4 

X  PE4 

43 

ETHNICITY 

BY 

PARENT'S  ED  (ASIAN  AMER,  COLL  GRAD) 

43 

E4 

X  PE_ 

44 

ETHNICITY 

BY 

PARENT'S  ED  (ASIAN  AMER,  UNKNOWN) 
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Table  B.25 
(continued) 


VARIABLE 

LABEL 

DESCRIPTION 

44 

MA,<MG 

45  MODAL  AGE,  LESS  THAN  MODAL  GRADE 

45 

MA,MG 

46  MODAL  AGE,  MODAL  GRADE,  MISSING 

46 

MA,>MG 

47  MODAL  AGE,  GRFATER  THAN  MODAL  GRADE 

47 

>MA,MG 

48  GREATER  THAN  MODAL  AGE,  MODAL  GRADE 

48 

SCH  TYPE 

A 9  SCHOOL  TYPE  (NOT  PUBLIC) 

49 

ASK  SW? 

50  FAMILY  ASKS  ABOUT  SCHOOLWORK  (ALMOST  EVERY  DAY) 

50 

PRESCHl 

51  WENT  TO  PRESCHOOL  (YES) 

51 

#PARENT1 

52  SINGLE/MULTIPLE  PARENT  HOME  (MOTHER, FATHER  HOME) 

52 

MOTHER 

53  MOTHER  AT  HOME 

53 

MOWORK 

54  MOTHER  WORKS  OUTSIDE  (YES) 

54 

GRADES 

55  GRADES  IN  SCHOOL  (LINEAR) 

55 

PR0GRAM2 

56  HIGH  SCHOOL  PROGRAM  (COLLEGE  PREPARATORY) 

56 

PR0GRAM3 

57  HIGH  SCHOOL  PROGRAM (VOCATIONAL) 

57 

MATH 

58  NUMBER  OF  MATH  COURSES 

58 

SCIENCE 

59  NUMBER  OF  SCIENCE  COURSES 

59 

P0STSEC2 

60  TWO-YEAR  COLLEGE 

60 

P0STSEC3 

61  FOUR-YEAR  COLLEGE 

61 

HRS  WORK 

62  HOURS  OF  OUTSIDE  WORK 

62 

ENGL23 

63  TYPES  OF  ENGLISH  CLASS (ADVANCED&COLLEGE  PREP.) 

63 

ENGLISH5 

64  TYPES  OF  ENGLISH  CLASS (REMEDIAL) 

64 

MATH  Ql 

65  MATH    QUANTILE  (LINEAR  -1,0,1) 

65 

SCI  Ql 

66  SCIENCE    QUANTILE  (LINEAR  -1,0,1) 
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Table  B.26 


Estimated  Effects  for  Science  Conditioning  Variables,  Grade  11/Age  17 


SUBSCALE    SUBSCALE    SUBSCALE    SUBSCALE  SUBSCALE 


1 

2 

3 

4 

5 

1 

0. 

043593 

0. 

063372 

0. 

106473 

0. 

352457 

-0. 

231871 

2 

-0. 

161091 

-0. 

289228 

-0. 

099476 

-0. 

716864 

-0. 

460884 

3 

-0. 

36247^j 

-0. 

338893 

-0. 

173738 

-0. 

620345 

-0. 

432612 

4 

-0. 

230492 

^0. 

553346 

-0. 

453106 

-0. 

383723 

0. 

004074 

5 

0. 

257795 

0. 

697624 

-0. 

327383 

-0. 

135944 

0. 

107313 

6 

0. 

181303 

0. 

155107 

0. 

141684 

0. 

207503 

0. 

253035 

7 

0. 

096863 

0. 

059891 

0. 

029983 

0. 

038749 

0. 

158398 

8 

-0. 

011213 

0. 

047387 

0. 

017864 

0. 

015855 

-0. 

005260 

9 

-0. 

004681 

0. 

027787 

0. 

082975 

0. 

007021 

0. 

075262 

10 

0. 

020535 

-0. 

066603 

0. 

030213 

0. 

049925 

0. 

071285 

11 

0. 

072452 

-0. 

066689 

-0. 

064837 

-0. 

055517 

0. 

161335 

12 

0. 

179656 

0. 

031182 

0. 

005316 

0. 

063548 

0. 

265283 

13 

0. 

149746 

0 

034793 

0 

079944 

0. 

106435 

0. 

240836 

14 

-0. 

022815 

-0. 

186059 

-0 

092720 

0. 

085932 

0. 

302199 

15 

0. 

131464 

0, 

142492 

0. 

106207 

0. 

094887 

0. 

129853 

16 

0. 

124030 

0. 

134147 

0 

123985 

0. 

114555 

0. 

156846 

17 

-0. 

001005 

-0. 

082887 

-0 

028116 

-0. 

038275 

-0. 

041108 

18 

-0 

004810 

0 

006694 

-0 

002735 

0 

000991 

0. 

000820 

19 

-0 

108159 

-0 

699705 

-0 

340852 

-0 

523118 

-0, 

079568 

20 

-0 

007321 

-0 

596946 

-0 

197045 

-0 

458036 

-0 

056911 

21 

-0 

009901 

-0 

002791 

-0 

010068 

-0 

027996 

0, 

009185 

22 

-0 

159796 

0 

179664 

0 

004145 

0 

048460 

-0, 

213472 

23 

-0 

124145 

-0 

547254 

0 

023533 

-0 

316701 

-0. 

044378 

24 

0 

144437 

-0 

232710 

-0 

081140 

-0 

010936 

0 

149002 

25 

-0 

232195 

-0 

111559 

-0 

039114 

-0 

001650 

-0 

198142 

26 

-0 

001776 

0 

019039 

-0 

.020661 

-0 

029408 

-0 

054646 

27 

-0 

.069033 

-0 

.124319 

-0 

.079437 

-0 

088034 

-0 

073965 

28 

-0 

.014637 

-0 

.000283 

0 

.003928 

-0 

.024071 

0 

030897 

29 

0 

.032172 

-0 

.048967 

0 

.132354 

0 

.195481 

0 

.059742 

30 

0 

.169771 

0 

.171371 

0 

.215543 

0 

.141315 

0 

.245622 

31 

-0 

.311658 

-0 

.486529 

0 

.106844 

-0 

.457785 

0 

.166015 

32 

-0 

.128565 

-0 

.165027 

-0 

.153785 

-0 

.012574 

-0 

.176237 

33 

-0 

.040215 

-0 

.121918 

-0 

.110535 

-0 

.107906 

-0 

.213731 

34 

-0 

.106096 

-0 

.105798 

-0 

.210725 

0 

.024428 

-0 

.075922 

35 

0 

.037949 

0 

.049074 

-0 

.386862 

-0 

.077424 

-0 

.222369 

36 

-0 

.085326 

0 

.102463 

0 

.126362 

-0 

.133838 

-0 

.422317 

37 

-0 

.001074 

0 

.332671 

0 

.180225 

-0 

.024222 

-0 

.299538 

38 

-0 

.119561 

-0 

.051860 

0 

.131508 

-0 

.004487 

-0 

.171454 

39 

0 

.046925 

0 

.202801 

0 

.344202 

-0 

.172031 

-0 

.488798 

40 

-0 

.292711 

-0 

.706969 

0 

.148923 

0 

.220247 

-0 

.969625 

41 

-0 

.138596 

-0 

.565604 

0 

.149780 

-0 

.231619 

-0 

.312820 

42 

-0 

.045111 

-0 

.370719 

0 

.320102 

0 

.355094 

-0 

.221272 

43 

-0 

.789749 

-0 

.553608 

0 

.192032 

-0 

.320634 

-0 

.824142 

44 

-0 

.141423 

-0 

.195838 

-0 

.267417 

-0 

.184574 

-0 

.115384 

-507- 


ERIC 


553 


Table  B.26 
(continued) 


SUBSCALE     SU3SCALE    SUBSCALE     SUBSCALE  SUBSCALE 


1 

X 

2 

3 

4 

5 

M- J 

W  a 

-0 . 

-0  . 

051633 

0 . 

007075 

0 . 

039927 

*+D 

-0 . 

1 00688 

-0 . 

018166 

0 . 

038639 

-0. 

053074 

LI 

-0 

-0 

91 97S7 

-0 . 

222941 

-0 . 

102418 

-0, 

100814 

M-O 

111  604 

-0 . 

130452 

-0 . 

115449 

-0 . 

179906 

-0 . 

111902 

-0 . 

039574 

-0 . 

046300 

-0 . 

025739 

-0. 

017585 

-0. 

048514 

0 . 

003232 

0 . 

1 0888S 

X  \7  U  (J  (J  ^ 

-0. 

004211 

-0 . 

009670 

-0. 

002218 

SI 

n 

091  1  L(\ 

-0 . 

014706 

0 . 

027883 

-0 . 

024738 

-0 . 

027867 

S9 

-0 

\j  % 

X  o  <c 

~n 
\j  • 

1 00S09 

-0 

105426 

0 . 

016892 

0 

039096 

-n 

nnfti 99 

O  X  <C  7 

-0 , 

036518 

0 

004197 

-0 

032206 

-0 

040905 

n 

0 

222743 

0 

213212 

0 

150295 

0 

156734 

0 

165642 

0 

138009 

0 

135968 

0 

134767 

0 

152624 

56 

-0 

008978 

0 

001769 

-0 

023027 

0 

.066621 

-0 

009418 

57 

0 

059011 

0 

.100075 

0 

095915 

0 

.103744 

0 

.084779 

58 

0 

.084300 

0 

.216691 

0 

.096578 

0 

.108890 

0 

,100083 

59 

-0 

.015932 

0 

.011491 

0 

.008584 

-0 

.041987 

-0 

.024621 

60 

0 

,092601 

0 

.191483 

0 

.087915 

0 

.013643 

0 

.086432 

61 

-0 

.002879 

-0 

,058677 

-0 

.078214 

-0 

.045114 

-0 

,017600 

62 

0 

.091970 

0 

.133598 

0 

.074131 

0 

.053897 

0 

.014220 

63 

-0 

.261116 

-0 

.286074 

-0 

.079385 

-0 

.087793 

-0 

.274558 

64 

-0 

.221969 

-0 

.157411 

-0 

.284129 

-0 

.281330 

-0 

,252945 

65 

-0 

-0 

.321484 

-0 

.280146 

-0 

,270467 

-0 

.255174 

-508- 

5CJ 


Table  B.27 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1976-77:    Age  9 


VARIABLE 

LABEL 

GAMMA 

1 

nVFRAT  T 

-0. 

582386 

i. 

2 

-0, 

113A65 

3 

FTHNTC2 

-1 

027636 

nR<sFR\7Fn   FTHMTTTTV  ^UTAri^\ 

4 

ETHNIC 3 

-0 

588893 

4 

nR<?FRVFn  FTHMTTTTY  ^HT<IPAMTr^ 

5 

ETHNIC4 

-0 

365009 

5 

nR^^FRVFn  FTHNTPTTY   ^A<?TAN  AMFRTTAM^ 

6 

ST0C2 

0 

.732791 

6 

<JT7F  ANH  TYPF  OF  mMMTTTJTTY    /'HTPH  MFTP^^ 

7 

ST0C3 

0 

.404707 

7 

^T7F   AUn  TYPF   OF  fnMMTThJTTV    ^tJOT  UTPU   OP   T  nuN 

8 

RFGT0N2 

-0 

.167073 

Q 

xV.EiVjXWii       OUU  Xni!*fliD  X  ^ 

9 

REGIONS 

0 

.110589 

9 

REGION  C CENTRAL 

10 

REGI0N4 

0 

.098635 

10 

REGION  (WEST) 

11 

PARED2 

0 

.290650 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

.498876 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.402642 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

.121465 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.614894 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.454198 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

.215899 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

.482111 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

SCH.PRIV 

0 

.133647 

19 

SCHOOL  TYPE  (PRIVATE) 

ERIC 


-509- 

,  561 


Table  B.28 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1976-77:    Age  13 


T7AD  T  A  TIT  T? 

ESTIMATED 

EFFECT 

DESCRIPTION 

1 

r\T7T?D  ATT 

757367 

1 

JL 

nVFRATT   CONS'^'ANT  '1'   FOR  EVERYONE 

2 

-  n 

JL\J  1.  ^  1.  1 

9 
Z 

O  El  A.    \  r  EJ.  LriJ-ii-i  _/ 

3 

T^T  TXT  T  O 

ETHNIC z 

-  n 

i  I  yJO  I  I 

o 

\JJD^Iljt\vCjU    .J_«  X  nil  X  \m»  X  JL  A     \  x»j_j0.v>xv/ 

i 

ElHNICi 

-  u  . 

J  D  D  y  KJ  KJ 

/, 

nR<;FR\7Fn  FTHlSUrTTY  ^'HTSPANTC'i 

c 

E 1 HN I C4 

-0 . 

218238 

t; 

nR<;FP\7Fn  FTHNTrTTY  ^ASIAN  AMERICAN"^ 

c 

0 

biOO/ 

0 

333642 

o 

<;t7F  and  type  of  community  (HIGH  METRO) 

7 

b  iOO  J 

n 

164775 

7 

<?T7F  AND  TYPF  OF  COhlMUNITY  CNOT  HIGH  OR  LOW) 

o 
o 

\j 

204287 

Q 
o 

RFHTON  (  SOUTHEAST 

£\,Xj\J  X\_/Ll      ^  iJ          XAlJ-i**.iJ  X  / 

Q 

-0 

034224 

Q 

y 

PFHTON   ^TIFNTRAT  ^ 

1  n 

-0 

.111973 

1  0 

REGION  (WEST) 

11 

PARED2 

0 

.185109 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

0 

.433452 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.483972 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

TaRED^ 

-0 

.033020 

14 

PARENTS  EDUCATION  (MISSING,  I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.537162 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.593768 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

.179763 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

.414808 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

SCH.PRIV 

0 

.162110 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 

-510- 


Table  B.29 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1976-77:    Age  17 


\7APTART  T7 
V  rVJx  X  riiJ  1j  IL 

ESTIMATED 

T  AUT7T 

T7FFT7rT 
nr  r  JL 

■ni?  cr'D  T  TJTTr^M 
UhDUKxr ixUN 

1 

JL 

0 . 

X  O  it.  H-  _^  J 

1 

X 

n\717PATT    mtJCTAMT    '1  '    T?nD  T7\7T7DVnM17 

9 

-0 , 

J  /  O  J  VJ  / 

9 

QTTY  /^?T?>JrATC^ 

n  X  nix  X  o 

-0 . 

^  \/  J  U  \/ 1> 

UiiDriKVriU  HinJNxUxil  (^jtSLAUJN.^ 

A 
*+ 

n  X  niN  X  w  3 

0*4X  J  O  3 

/, 

nUCT7D\7T7'n    T7TUKrTr'TTV  /UTCDAXTT^'^ 

Ui3oriKVr.U  rjinJNXUXxx    v^nXori?\IN xU^ 

D 

T7TUMTr'/t 

£jirlINxO'+ 

-0 , 

C  TTDTTTTT^     170^11X1  T .  T'XT     /ACT  AM     A  lJ'T7r>  T A  XT  \ 

UlibliKVliD  EiHNICxlY   (ASIAW  AMERIOAN) 

D 

QTnr'9 

yj 

X  ^  o  o  xo 

o 

7 

yj 

U  O  O    O  X 

7 

biZhi  AND  TYrE  Ur  COMMUNITY  (NOi  HIGH  OR  LOW) 

Q 
O 

\j 

X  iL.  O  O  /  vJ 

Q 

\j 

REGION  (oOUiHEAbi) 

Q 

JxHioXUiN  J 

-0 

026421 

9 

DTJ/^TnU    /r'T7MTDAT  ^ 

10 

-0 

107374 

10 

IxIlikjXUiN  ^WILDX^ 

11 

PARED2 

0 

132593 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAB) 

12 

PARED3 

0 

.306682 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.529397 

13 

PARENTS  EDUCATION  (COLLEGE  GRAB) 

14 

PARED_ 

-0 

.084897 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.650818 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.164101 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

.298438 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

.484623 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

SCH.PRIV 

0 

.084481 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 

ERIC 


-511- 

563 


Table  B.30 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1981-82:    Age  9 


VARIABLE 

ESTIMATED 

T  A  T>  t?T 

ty'p'ptyr'T 

1 

OVERALL 

-  u  . 

1  LLfA 

1 

m7T?DATT   rnNC!TAMT  '1'   FOR  EVERYONE 

2 

GENDER2 

n 

-  u . 

U  JUU  / 

O 

z. 

3 

ETHNIC* 

n 

O  /  ^ 

nRQT?P\7T?n  FTHNTTTTY  ^BTACK'i 

UOO  Il<£\.  V  E<L/    I_<  XriM  X  v-»  i  X  A     \  A-PJ-tTXvrv/ 

t?«T«t  TXT  T  O 

ETHNIC3 

-0 

O  ^  J.  ^  \J  ^ 

A 

nRCiT?P\7Fn  FTHNTflTTY  ('HTSPANIC'i 

5 

nR<?FRVFD  ETHNICITY  (ASIAN  AMERICAN) 

6 

SiOOz 

0 

SI  9993 

D 

<?T7F  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0 

J.  ^  ^  ^  ^ 

7 

<^T7F  AND  TYPE  OF  COMrfUNITY  (NOT  HIGH  OR  LOW) 

o 
0 

RLGIONZ 

-0 

007397 

Q 
O 

RFflTON  (<?OUTHEAST) 

Q 

0 

012891 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

-0 

047799 

10 

REGION  (WEST) 

11 

PARED2 

0 

.329554 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.514058 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

.174065 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.702701 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.723683 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

H0MEITM3 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

H0MEITM4 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

SCH.PRIV 

-0 

.051738 

19 

SCHOOL  TYPE  (PRIVATE) 

ERIC 


rH2- 


Table  B.31 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1981-82:    Age  13 


VARIABLE 

ESTIMATED 

LABEL 

EFFECT 

DESCRIPTION 

1 

OVEEIALL 

-0. 

560294 

1 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

-0. 

268114 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

-0 

709938 

3 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNICS 

-0 

496880 

4 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0 

009130 

5 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

ST0C2 

0 

474841 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0 

194103 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

-0 

125149 

8 

REGION  (SOUTHEAST) 

9 

REGIONS 

0 

038679 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

0 

002386 

10 

REGION  (WEST) 

11 

PARED2 

0 

228734 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

513646 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

582109 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

064964 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.603198 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

848736 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

H0MEITM3 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

H0MEITM4 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

SCH.PRIV 

0 

.071106 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 

ERIC 


-513- 

565 


Table  B.32 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1981-82:    Age  17 


VARIABLE 

ESTIMATED 

LABEL 

EFFECT 

DESCRIPTION 

1 

OVERALL 

0. 

247207 

1 

OVERALL  CONSTANT  '1'   FOR  EVERYONE 

2 

GENDER2 

-0. 

441003 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

-1. 

038798 

3 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC 3 

-0. 

650045 

4 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0 

384856 

5 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

ST0C2 

0, 

359896 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0 

210354 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

0 

024158 

8 

REGION  (SOUTHEAST) 

9 

REGI0N3 

0 

062773 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

0 

.015087 

10 

REGION  (WEST) 

11 

PARED 2 

0 

.138555 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

0 

.429813 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.617670 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

-0 

.190094 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.624983 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.098699 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

H0MEITM3 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

H0MEITM4 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

17 

SCH.PRIV 

-0 

.009154 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 

ERIC 


-514- 


sec 


Table  B.33 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 
1986:    Age  9,  Bridge  A 

VARIABLE  ESTIMATED 

LABEL  EFFECT  DESCRIPTION 

1  OVERALL  -0.379284    1  OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2  GENDER2  -0.161779    2  SEX  (FEMALE) 

3  ETHNIC2  -0.669729    3  OBSERVED  ETHNICITY  (SLACK) 

4  ETHNIC3  -0.489115    4  OBSERVED  ETHNICITY  (HISPANIC) 

5  ETHNIC4  -0.102341     5  OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6  ST0C2  0.703567    6  SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7  ST0C3  0.464462     7  SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8  REGI0N2  -0.115330    8  REGION  (SOUTHEAST) 

9  REGI0N3  -0.041812    9  REGION  (CENTRAL) 

10  REGI0N4  -0.090698  10  REGION  (WEST) 

11  PARED2  0.169671  11  PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12  PARED3  0.429293  12  PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13  PARED4  0.385039  13  PARENTS  EDUCATION  (COLLEGE  GRAD) 

14  PARED_  0.122934  14  PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15  <  MODALG  -0.629587  15  MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16  >  MODALG  0.403331  16  MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17  H0MEITM3  0.258453  17  ARTICLES  IN  HOME  (YES  TO  3) 

18  H0MEITM4  0.349804  18  ARTICLES  IN  HOME  (YES  TO  4) 

19  SCH.PRIV  -0.023166  19  SCHOOL  TYPE  (PRIVATE) 


-515-  re- 


Table  B.34 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 
1986:    Age  9,  Bridge  B 


VARIABLE 

ESTIMATED 

LABEL 

EFFFCT 

DESCRIPTION 

1 

OVERALL 

-0. 

769386 

1 

OVERALL  CONSTANT  '1'  FOR  EVERYONE 

2 

GENDER2 

-0. 

038277 

2 

SEX  (FEMALE) 

3 

ETHNIC2 

-0. 

739842 

3 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNICS 

-0. 

586415 

4 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

103627 

5 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

ST0C2 

0. 

891562 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0. 

548884 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

-0. 

129702 

8 

REGION  (SOUTHEAST) 

9 

REGIONS 

0 

011742 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

-0 

119800 

10 

REGION  (WEST) 

11 

PARED2 

-0 

004067 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

0 

.271551 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.218143 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

.045211 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.871979 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.411836 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

.219618 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

.336150 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

SCH.PRIV 

-0 

.074180 

19 

SCHOOL  TYPE  (PRIVATE) 

ERIC 
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Table  B.35 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 
1986:    Age  13,  Bridge  A 


VARIABLE 

ESTIMATED 

LABEL 

EFFECT 

DESCRIPTION 

1 

OVERALL 

-0 

693683 

1 

OVERALL  CONSTANT  '1'   FOR  EVERYONE 

2 

GENDER2 

-0. 

219859 

2 

SEX  (FEMALE) 

3 

ETHNIG2 

-0. 

691137 

3 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC3 

-0. 

492987 

4 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

099713 

5 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

ST0C2 

0 

414012 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0 

246599 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REGI0N2 

-0 

033529 

8 

REGION  (SOUTHEAST) 

9 

REGIONS 

-0 

023863 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

-0 

078740 

10 

KEGION  (WEST) 

11 

PARED2 

0 

138635 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED3 

0 

329713 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARZD4 

0 

.428845 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

.054985 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.458554 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.267685 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

.201938 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

.340467 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

SCH.PRIV 

0 

.010178 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 

o 

ERIC 
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Table  B.36 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 
1986:    Age  13,  Bridge  B 

i 


VARIABLE 

ESTIMATED 

LABEL 

EFFECT 

DESCRIPTION 

1 

OVERALL 

-0. 

915927 

1 

OVERALL  CONSTANT  '1'   FOR  EVERYONE 

2 

GENDER2 

-0. 

198725 

2 

SEX  (FEMALE) 

3 

ETHNIC 2 

-0. 

639180 

3 

OBSERVED  ETHNICITY  (BLACK) 

4 

ETHNIC 3 

-0, 

289440 

4 

OBSERVED  ETHNICITY  (HISPANIC) 

5 

ETHNIC4 

-0. 

147956 

5 

OBSERVED  ETHNICITY  (ASIAN  AMERICAN) 

6 

ST0C2 

0. 

363875 

6 

SIZE  AND  TYPE  OF  COMMUNITY  (HIGH  METRO) 

7 

ST0C3 

0, 

321760 

7 

SIZE  AND  TYPE  OF  COMMUNITY  (NOT  HIGH  OR  LOW) 

8 

REG I ON 2 

-0 

176897 

8 

REGION  (SOUTHEAST) 

9 

REGI0N3 

-0 

010910 

9 

REGION  (CENTRAL) 

10 

REGI0N4 

-0 

224578 

10 

REGION  (WEST) 

11 

PARED2 

0 

.101594 

11 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAB) 

12 

PARED3 

0 

.414792 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

.444902 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

0 

.128516 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

.672904 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

.136297 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

,227592 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

.398187 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

19 

SCH.PRIV 

0 

.188192 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 

ERIC 
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Table  B.37 


Estimated  Effects  for  Science  Trend  Conditioning  Variables 

1986:    Age  17 


VARIABLE 

ESTIMATED 

LABEL 

FFFFCT 

n  R  Q  P  D  T  PTT  nXT 

1 

OVERALL 

-0, 

317737 

U  V  ILXxn.J-«l^   OUlNDlAiNl       JL       rUK    Ei  V  lilvlUJNEi 

2 

GENDER2 

-0, 

307810 

2 

^FY  /FFMATR^ 

3 

ETHNIC  2 

-0, 

624208 

3 

4 

ETHNIC  3 

-0, 

361968 

4 

OR55ERVED  FTHNTHTTV  /'HT<>PAUTrN 

5 

ETHNIC4 

0, 

018743 

5 

OR55FRVFD  FTHMTflTTY   ^A^^TAN  AMRRTTAMN 

\jutj  Luis,  y/  LuL/    c»  i  xlltj  X    X  i  i     ^r\.o  xnxi    ^tti^EjIvX  wriiM  j 

6 

ST0C2 

0, 

434584 

6 

^T7F  AKfn  TYPF  OF  POMMTrMTTV   ^UJnU  MRTunN 
cixiu   xxiTiL  Kjr   oufiriujNxxx    ^rlxvjxl  niLiKU^ 

7 

ST0C3 

0, 

443949 

7 

CT7F   AMfi  TYPR   OF  mMMTrwTTV    /^MOT  UTr«u   no   T  nTT\ 

8 

REGI0N2 

-0, 

089567 

8 

9 

REGI0N3 

0 

017682 

9 

REGION  ('CFNTRAT 

10 

REGI0N4 

-0, 

165501 

10 

REGION  (WEST) 

11 

PARED2 

0, 

214393 

il 

PARENTS  EDUCATION  (HIGH  SCHOOL  GRAD) 

12 

PARED 3 

0 

516285 

12 

PARENTS  EDUCATION  (POST  HIGH  SCHOOL) 

13 

PARED4 

0 

609179 

13 

PARENTS  EDUCATION  (COLLEGE  GRAD) 

14 

PARED_ 

-0 

330536 

14 

PARENTS  EDUCATION  (MISSING,   I  DON'T  KNOW) 

15 

<  MODALG 

-0 

468667 

15 

MODAL  GRADE  (LESS  THAN  MODAL  GRADE) 

16 

>  MODALG 

0 

160284 

16 

MODAL  GRADE  (GREATER  THAN  MODAL  GRADE) 

17 

H0MEITM3 

0 

219045 

17 

ARTICLES  IN  HOME  (YES  TO  3) 

18 

H0MEITM4 

0 

405511 

18 

ARTICLES  IN  HOME  (YES  TO  4) 

SCH.PRIV 

0 

301300 

19 

SCHOOL  TYPE  (PRIVATE) 

SCH.MISS 

20 

SCHOOL  TYPE  (MISSING) 
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Appendix  C 
WARM  VARIABLES  TABLES 


Bruce  Kaplan 
Educational  Testing  Service 


Appendix  C  contains  12  tables  showing  the  relationship  between  NAEP 
items  and  WARM  scores  for  reading  (Tables  C.l  through  C.4),  mathematics 
(Tables  C.5  through  C.8),  and  science  (Tables  C.9  through  C.12).     Certain  of 
these  WARM  scores  were  used  in  the  subject  area  reports.     Other  WARM 
variables  were  excluded  from  the  reports  because  of  their  minimal 
relationships  with  the  proficiency  values. 

The  first  three  tables  for  each  learning  area  (one  table  for  each 
grade/age)  contain  the  items  used  to  construct  each  WARM  score.     Each  WARM 
variable  and  short  description  is  followed  by  a  list  of  the  NAEP  items  used 
in  its  construction.     Each  NAEP  item  is  shown  with  its  corresponding  short 
description,  the  block  in  which  it  appears,  and  its  location  within  the 
block. 

The  WARM  variable  names  (for  example,  RWlAx)  are  created  from: 

the  initial  letter  of  the  learning  area  (R  for  reading,  M 
for  mathematics,  or  S  for  science); 

the  WARM  variable  identifier  (W) ; 

the  cohort  number  (1  for  grade  3/age  9,  2  for  grade  7/age 
13,  or  3  for  grade  11/age  17); 

the  WARM  score  identifier  (A,  B,  C,  D,  E,  F,  G,  or  H) ;  and 

the  plausible  value  identifier  (x) ,  representing 
plausible  values  1  though  5. 

The  remaining  tables  show  how  NAEP  item  responses  were  rescored  to 
calculate  WARM  scores  for  reading  (Table  C.4),  mathematics  (Table  C.8),  and 
science  (Table  C.12).     All  items  used  to  construct  WARM  scores  for  a  given 
learning  area  are  listed  in  NAEP  ID  order.     If  an  item  was  u.^od  to  construct 
a  particular  WARM  score  for  one  or  more  cohorts,  the  appropriate  WARM  score 
identifier  (A  through  H)  appears  in  the  appropriate  COHORT  column(s)  (1,  2, 
or  3) .     Each  item  in  a  learning  area  was  used  to  construct  only  one  WARM 
score  for  a  given  cohort. 

The  right  side  of  Tables  C.4,  C.8,  and  C.12  shows  how  each  NAEP  item  was 
rescored  to  calculate  the  WARM  score.     The  column  headings  indicate  the 
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original  item  responses  (1  through  8,  and  MISSING).     Listed  below  these 
headings  are  the  values  to  which  the  responses  were  recoded. 

For  example,  in  Table  C.4,  reading  item  S003301  was  used  to  calculate 
reading  WARM  score  C  for  cohort  2  (grade  7/age  13)  and  cohort  3  (grade  11/age 
17).     Original  item  responses  1,  2,  and  3  were  respectively  recoded  as  1,  3, 
and  5. 


Table  C.l 
Reading  WARM  Variables 
Grade  3/Age  9 

BLOCK  ITEM 


RWlAx  -  Study  habits 


O  A  A  /.  "T  A 1 

HOW  OFTEN  DOES  TEACHER  LIST  OF  QUESTS  AS  YOU  READ 

R2 

4 

HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  READ  FASTER 

R2 

6 

SO051O2 

HOW  OFTEN  WHEN  STUDY  FOR  TEST:  TAKE  NOTES  ON  READ 

Rl 

2 

oUUD  iU J 

HOW  OrihN  WHhN  STUDY  FOR  TEST:  MAKE  OUTLINES 

Rl 

3 

RWlBx  -  Reading  habits 

nUW  UrilLiM  uu   lUU   ihLL  A  rKihINU  Ai5UUi  A  GOOD  BOOK 

R3 

3 

nUW  urilLIN  uu    lUU   orCiINU    lUUK  UWJN  flUINCii   UN  15U0Ko 

n  0 
RJ 

5 

tlUW   uriCilN   uu    lUU   Kr*AU  dUUIs.  D/\oiLU  UIN  WUVic*   YUU  oAW 

0 

Hnu  nirTT?Ki  nn  vhtt  dp  ah  unnvQ  rv  am  ATTTunD  vhtt  t  tvt? 

nww    ur            uu    lUU    i\.J!»A.U    IjUUIxo    D1    AJN   AUiilUK.    lUU  LillxlL 

p'i 

/ 

i 

S007202 

HOW  OFTEN  YOU  TALK  W/FRIEND  ABOUT  SOMETHING  READ 

R4 

2 

nuw  ur  lEiiN  uu   luu   rs.c«AJj  rAKi   Ur   r<£<Wo rHir UK 

kd 

JL 

S007303 

HOW  OFTEN  DO  YOU  READ  PART  OF  MAGAZINE 

R5 

3 

S007305 

HOW  OFTEN  DO  YOU  READ  BIOGRAPHY 

R5 

5 

I\.nJ.UX          ±  Llo  UJ.  UC  UXOIl 

UTTH   MFU  RFAHTMn   HOTJ  fiPTFM   TT7APHT?P    PHTKIT  PAPH  TJriPn 
wj.j.n  nctW   iviLriLf nuw   ur           1  J!«f\.uni!<i\.  irUilNi   ilnJvU  WUi\U 

1 
1 

S004602 

WITH  NEW  READING  HOW  OFTEN  TEACHER  PREVIEW  READING 

R2 

2 

S004603 

WITH  NEW  READING  HOW  OFTEN  TEACHER  READ  PART  ALOUD 

R2 

3 

S004702 

HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  FIND  MAIN  IDEA 

R2 

5 

cnnsi m 

nuw  uriCiiN  wmLiN  oiuui  rUK  ilSoi.   Kji»A.U  uVlsK  MAiCiKiAL 

Ri 

i 

SOOSl 04 

w>  \J  \J  ~/  J.  V-r 

nuw  uridLi   wnc*i.i   oxui/i   ruK.   xilol,    v^UjI^   LVi    \  ti^ a.  1  DUUi\ 

P  1 

S005203 

HOW  OFTEN  DO  YOU  WORK  IN  A  WORKBOOK 

Rl 

9 

KWlux      inaiviauai  ireaQing 

S003501 

HOW  OFTEN  DO  YOU  READ  FOR  FUN  ON  YOUR  OWN  TIME 

R3 

2 

SO05202 

HOW  OFTEN  DO  YOU  READ  ON  YOUR  OWN  IN  SCHOOL 

Rl 

8 

S007301 

HOW  OFTEN  DO  YOU  READ  PART  OF  NOVEL  OR  STORY 

R5 

1 

RWlEx  -  Reading  aloud 

S004401 

HOW  OFTEN  DOES  SOMEONE  READ  ALOUD  TO  YOU 

Rl 

10 

S004402 

HOW  OFTEN  DO  YOU  READ  ALOUD  TO  SOMEONE 

Rl 

11 

S005201 

HOW  OFTEN  DO  YOU  READ  ALOUD  IN  SCHOOL 

Rl 

7 
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Table  C.2 


Reading  WARM  Variables 
Grade  7/Age  13 


BLOCK  ITEM 


S005102 
S005103 
S005104 
S005105 
S005106 
S005301 
S005302 
S0053O3 
S005304 


RW2AX  -  Study  habits 


HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 


WHEN  STUDY  FOR  TEST: 
WHEN  STUDY  FOR  TEST: 
WHEN  STUDY  FOR  TEST: 
WHEN  STUDY  FOR  TEST: 
WHEN  STUDY  FOR  TEST: 


TAKE  NOTES  ON  READ 
MAKE  OUTLINES 
QUES  IN  TEXTBOOK 
ANSWER  OWN  QUESTNS 
QUESTION  OTHERS 


GO  TO  LIBRARY  TO  READ  ON  OWN 
GO  TO  LIBRARY  TO  LOOK  UP  FACT  FOR  SCHOOL 
GO  TO  LIBRARY  TO  FIND  BOOKS  FOR  HOBBIES 
GO  TO  LIBRARY  FOR  QUIET  PLACE  TO  READ 


Rl 
Rl 
Rl 
Rl 
Rl 
Rl 
Rl 
Rl 
Rl 


2 
3 
4 
5 
6 
10 
11 
12 
13 


RW2Bx  -  Reading  habits 

5004601  WITH  NEW  READING  HOW  OFTEN  TEACHER  POINT  HARD  WORD  R2  1 

5004602  WITH  NEW  READING  HOW  OFTEN  TEACHER  PREVIEW  READING  R2  2 

5004603  WITH  NEW  READING  HOW  OFTEN  TEACHER  READ  PART  ALOUD  R2  3 

5004701  HOW  OFTEN  DOES  TEACHER  LIST  OF  QUESTS  AS  YOU  READ  R2/R4  4/1 

5004702  HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  FIND  MAIN  IDEA  R2/R4  5/2 

5004703  HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  READ  FASTER  R2  6 


RW2Cx  -  Instruction 


S003301 
S004301 
S004302 
S004303 
S004307 
S004309 
S004311 
S004503 
S005201 
S005202 
S005305 
S007201 
S007202 


WHAT  KIND 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 
HOW  OFTEN 


OF  READER  ARE  YOU 
DO  YOU  READ  A  STORY  OR  NOVEL 
A  POEM 
A  PLAY 
A  BIOGRAPHY 

A  BOOK  ABOUT  OTHER  TIMES 
SONG 


DO  YOU  READ 
DO  YOU  READ 
DO  YOU  READ 
DO  YOU  READ 

DO  YOU  READ  WORDS  OF  A 
DOES  FAMILY  READ  BOOKS 
DO  YOU  READ  ALOUD  IN  SCHOOL 
DO  YOU  READ  ON  YOUR  OWN  IN  SCHOOL 
GO  TO  LIBRARY  TO  TAKE  OUT  BOOKS 
YOU  TALK  AT  HOME  ABOUT  SOMETHING  READ 
YOU  TALK  W/FRIEND  ABOUT  SOMETHING  READ 


Rl 
R3 
R3 
R3 
R3 
R3 
R3 
R3 
Rl 
Rl 
Rl 
R6 
R6 


19 
1 
2 
3 
7 
9 

11 

14 
7 
8 

14 
1 
2 
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Table  C.2 
(continued) 


BLOCK  ITEM 


RW2DX  -  Individual  reading 

800430^  HOW  OFTEN  DO  YOU  READ  A  NEWSPAPER 

S004305  HOW  OFTEN  DO  YOU  READ  A  MAGAZINE 

S004310  HOW  OFTEN  DO  YOU  READ  A  SPORTS  BOOK 

5004501  HOW  OFTEN  DOES  FAMILY  READ  NEWSPAPERS 

5004502  HOW  OFTEN  DOES  FAMILY  READ  MAGAZINES 
S004504  HOW  OFTEN  DOES  FAMILY  READ  RECIPES 

5005402  HOW  OFTEN  DO  YOU  READ  A  NEWS  MAGAZINE 

5005403  HOW  OFTEN  DO  YOU  READ  NEWSPAPER  NOT  COMICS  OR  SPRT 


R3 
R3 
R3 
R3 
R3 
R3 
Rl 
Rl 


4 
5 
10 
12 
13 
15 
16 
17 
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Table  C.3 
Reading  WARM  Variables 
Grade  11/Age  17 


BLOCK 

ITEM 

RW3AX  -  Study  habits 

S005102 

HOW  OFTEN  WHEN  STUDY  FOR  TEST:  TAKE  NOTES  ON  READ 

Rl 

2 

Hnu  OFTPW  UHFN  <>TTTnY  FOR  TF^T'    MAKF  OlJTi'TNF^? 

Rl 

3 

HOU  OFTFN  WHFN  STUDY  FOR  TEST'  DUES  IN  TEXTBOOK 

Rl 

4 

cOOSl  OS 

HOU  OFTFN  WHEN  STUDY  FOR  TEST'  ANSWER  OWN  OUESTNS 

Rl 

5 

<;00S1  Ofi 

HOW  OFTFN  WHEN  STUDY  FOR  TEST'  QUESTION  OTHERS 

Rl 

6 

<;oos'^oi 

HOW  OFTFN  no  TO  T IRRARY  TO  READ  ON  OWN 

Rl 

10 

<;oos'^09 

HOW  OFTFN  GO  TO  T  IRRARY  TO  LOOK  UP  FACT  FOR  SCHOOL 

XlvW     v/X  X  1I<X^                  X  Vy      1.1  1  ir\  r~\.  ia  i       X  vy     X^VWXV     vy  X      X  X^\y  X     X  xy^x.  tj\yxivyvyxj 

Rl 

11 

cOOS'^O'^ 

HOW  OFTEN  GO  TO  LIRRARY  TO  FIND  BOOKS  FOR  HOBBIES 

Rl 

12 

S005?04 

HOW  OFTEN  GO  TO  LIBRARY  FOR  QUIET  PLACE  TO  READ 

Rl 

13 

RW3Bx  -  Reading  habits 

S004601 

WITH  NEW  READING  HOW  OFTEN  TEACHER  POINT  HARD  WORD 

R2 

1 

S004602 

WITH  NEW  READING  HOW  OFTEN  TEACHER  PREVIEW  READING 

R2 

2 

S004603 

WITH  NEW  READING  HOW  OFTEN  TEACHER  READ  PART  ALOUD 

R2 

3 

S004701 

HOW  OFTEN  DOES  TEACHER  LIST  OF  QUESTS  AS  YOU  READ 

XXxy  rt      X-/  X   X  X-iX^      Xy      XJ           X  XjX^x^XIX^Xx      Xj  JL       X      X-r  A.       x^  w  X^k^  ^           a              x,  x-r  w  xxx^a 

R2 

4 

S004701 

HOW  OFTEN  DOES  TEACHER  LIST  OF  GUESTS  AS  YOU  READ 

X.L\J  n       xy  X   X  X^Xv                 ALd            X  X^XXx^Xl  XjXx      Xj  JL       X       X-r  X.        x^x^       k^  ^  k^      a  Xk^       x  x-r  x^      ^x  ■  m 

R4 

1 

S004702 

HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  FIND  MAIN  IDEA 

XXx^n     xy  X  X  x^x^     xyxyx^ky     x       xxyxix-jxx     ^  x^xjxj    x&xyn      ^  x^     x  jlx^x^    xx4x<b>xi     «*■  x^  i  *j  x 

R2 

5 

S004702 

HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  FIND  MAIN  IDEA 

R4 

2 

S004703 

HOW  OFTEN  DOES  TEACHER  TELL  HOW  TO  READ  FASTER 

R2 

6 

RW3Cx  -  Instruv-tion 

S003301 

WHAT  KIND  OF  READER  ARE  YOU 

Rl 

19 

S004301 

HOW  OFTEN  DO  YOU  READ  A  STORY  OR  NOVEL 

R3 

1 

S004302 

HOW  OFTEN  DO  YOU  READ  A  POEM 

R3 

2 

S004303 

HOW  OFTEN  DO  YOU  READ  A  PLAY 

R3 

3 

S004307 

HOW  OFTEN  DO  YOU  READ  A  BIOGRAPHY 

R3 

7 

S004309 

HOW  OFTEN  DO  YOU  READ  A  BOOK  ABOUT  OTHER  TIMES 

R3 

9 

S004311 

HOW  OFTEN  DO  YOU  READ  WORDS  OF  A  SONG 

R3 

11 

S004503 

HOW  OFTEN  DOES  FAMILY  READ  BOOKS 

R3 

14 

S005201 

HOW  OFTEN  DO  YOU  READ  ALOUD  IN  SCHOOL 

Rl 

7 

S005202 

HOW  OFTEN  DO  YOU  READ  ON  YOUR  OWN  IN  SCHOOL 

Rl 

8 

S005305 

HOW  OFTEN  GO  TO  LIBRARY  TO  TAKE  OUT  BOOKS 

Rl 

14 

S007201 

HOW  OFTEN  YOU  TALK  AT  HOME  ABOUT  SOMETHING  READ 

R6 

1 

S007202 

HOW  OFTEN  YOU  TALK  W/FRIEND  ABOUT  SOMETHING  READ 

R6 

2 
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Table  C.3 
(continued) 


BLOCK 


ITEM 


RW3DX  -  Individual  reading 


5004304  HOW  OFTEN  DO  YOU  READ  A  NEWSPAPER  R3 

5004305  HOW  OFTEN  DO  YOU  READ  A  MAGAZINE  R3 
S004310  HOW  OFTEN  DO  YOU  READ  A  SPORTS  BOOK  R3 

5004501  HOW  OFTEN  DOES  FAMILY  READ  NEWSPAPERS  R3 

5004502  HOW  OFTEN  DOES  FAMILY  READ  MAGAZINES  R3 
S004504  HOW  OFTEN  DOES  FAMILY  READ  RECIPES  R3 

5005402  HOW  OFTEN  DO  YOU  READ  A  NEWS  MAGAZINE  Rl 

5005403  HOW  OFTEN  DO  YOU  READ  NEWSPAPER  NOT  COMICS  OR  SPRT  Rl 


4 
5 
10 
12 
13 
15 
16 
17 
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Table  C.4 
Derivation  of  Reading  WARM  Scores 


COHORT   RESPONSES  

123  12345678  MISS 


1 

S003301 

C 

C 

1 

3 

5 

M 

2 

S003501 

D 

5 

4 

3 

2 

1 

M 

3 

S003502 

B 

5 

4 

3 

2 

1 

M 

4 

S003504 

B 

5 

4 

3 

2 

1 

M 

5 

S003505 

B 

5 

4 

3 

2 

1 

H 

6 

S003506 

B 

5 

4 

3 

2 

1 

M 

7 

S004301 

C 

C 

5 

4 

3 

2 

1 

M 

8 

S004302 

C 

C 

5 

4 

3 

2 

1 

M 

9 

S004303 

C 

C 

5 

4 

3 

2 

1 

M 

10 

S004304 

D 

D 

5 

4 

3 

2 

1 

M 

11 

S004305 

D 

D 

5 

4 

3 

2 

1 

M 

12 

S004307 

C 

C 

5 

4 

3 

2 

1 

M 

13 

S004309 

C 

C 

5 

4 

3 

2 

1 

M 

14 

S004310 

D 

D 

5 

4 

3 

2 

1 

M 

15 

S004311 

C 

C 

5 

4 

3 

2 

1 

M 

16 

S004401 

E 

5 

4 

3 

2 

1 

M 

17 

S004402 

E 

5 

4 

3 

2 

1 

M 

18 

S004501 

D 

D 

5 

4 

3 

2 

1 

M 

19 

S004502 

D 

D 

5 

4 

3 

2 

1 

M 

20 

S004503 

C 

C 

5 

4 

3 

2 

1 

M 

21 

S004504 

D 

D 

5 

4 

3 

2 

1 

M 

22 

S004601 

C 

B 

B 

5 

4 

3 

2 

1 

M 

23 

S004602 

C 

B 

B 

5 

4 

3 

2 

1 

M 

24 

S004603 

C 

B 

B 

5 

4 

3 

2 

1 

M 

25 

S004701 

A 

B 

B 

5 

4 

3 

2 

1 

M 

26 

S004702 

C 

B 

B 

5 

4 

3 

2 

1 

M 

27 

S004703 

A 

B 

B 

5 

4 

3 

2 

1 

M 

28 

soo.sioi 

C 

5 

4 

3 

2 

1 

M 

29 

S005102 

A 

A 

A 

5 

4 

3 

2 

1 

M 

30 

S005103 

A 

A 

A 

5 

4 

3 

2 

1 

M 

31 

S005104 

C 

A 

A 

5 

4 

3 

2 

1 

M 

32 

S005105 

A 

A 

5 

4 

3 

2 

1 

M 

33 

S005106 

A 

A 

5 

4 

3 

2 

1 

M 

34 

S005201 

E 

C 

C 

5 

4 

3 

2 

1 

M 

35 

S005202 

D 

C 

C 

5 

4 

3 

2 

1 

M 

36 

S005203 

C 

5 

4 

3 

2 

1 

M 

37 

S005301 

A 

A 

5 

4 

3 

2 

1 

M 

38 

S005302 

A 

A 

5 

4 

3 

2 

1 

M 

39 

S005303 

A 

A 

5 

4 

3 

2 

1 

M 

40 

S005304 

A 

A 

5 

4 

3 

2 

1 

M 

ERIC 
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Table  C.4 
(continued) 


123  12345678  MISS 

41  S005305             CC  54321  M 

42  S005402             DD  54321  M 

43  S005403            DD  54321  M 

44  S007201  BCC  54321  M 

45  S007202  BCC  54321  M 

46  S0073O1  D  5       4        3        2        1  M 

47  S0073O2  B  5       4        3        2        1  M 

48  S007303  B  5       4        3        2        1  M 

49  S007305  B  5       4        3        2        1  M 


ERIC 
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Table  C.5 
Mathematics  WARM  Variables 
Grade  3/Age  9 


BLOCK  ITEd 

MWlAx  -  Enjoyment  of  math 

S205701      ARE  YOU  AS  GOOD  IN  MATH  AS  OTHERS  YOUR  AGE  M4  6 

5205901  TRUE  I  USUALLY  UNDERSTAND  CURRENT  MATH  LESSON  M5  1 

5205902  TRUE  I  AM  GOOD  WITH  NUMBERS  M5  2 

5205903  TRUE  DOING  MATH  MAKES  ME  NERVOUS  M5  3 

5205904  TRUE  MATH  IS  BORING  M5  4 

5205905  TRUE  I  AM  WILLING  TO  WORK  HARD  TO  DO  WELL  IN  MATH  M5  5 

5205906  TRUE  I  LIKE  MATHEMATICS  M5  6 

5206701  FEEL: LIKE  TO  PLAY  MATHEMATICS  GAMES  M7  1 

5206702  FEEL: HELPS  ME  TO  PLAY  MATHEMATICS  GAMES  M7  2 
S207101      FEEL:WOULD  LIKE  TO  WORK  AT  JOB  USING  MATH  M7  7 


MWlBx  -  Calculator  usage 

5206101  HOW  OFTEN  USE  A  CALCULATOR  IN  MATH  CLASS 

5206102  HOW  OFTEN  USE  A  CALCULATOR  IN  SCIENCE  CLASS 

5206103  HOW  OFTEN  USE  A  CALCULATOR  IN  OTHER  CLASSES 

5206104  HOW  OFTEN  USE  A  CALCULATOR  OUTSIDE  OF  SCHOOL 


M5 
M5 
M"" 

i  * 


7 
8 
9 
10 


MWlCx  -  Participation  in  math 

5207601  HOW  OFTEN  DO  YOU  LISTEN  TO  MATH  LESSON  EXPLAINED  M6  1 

5207602  HOW  OFTEN  DO  YOU  WATCH  MATH  TEACHER  WORK  PROBLEMS  M6  2 

5 20 7 603  HOW  OFTEN  DO  YOU  USE  MATH  WORKBOOK  OR  DITTO  SHEETS  M6  3 

5207604  HOW  OFTEN  DO  YOU  WORK  MATH  PROBLEMS  AT  BOARD  M6  4 
S207606      HOW  OFTEN  DO  YOU  WORK  MATH  PROBLEMS  ALONE  M6  6 


MWlDx  -  Gets  help  in  math 

S20550^  DO  SIBLINGS  REGULARLY  HELP  WITH  MATH  HOMEWORK 

S205503  DO  FRIENDS  REGULARLY  HELP  WITH  MATH  HOMEWORK 

5206703  FEEL: LIKE  GETTING  HELP  FROM  TEACHER  ON  MATH 

5206704  FEEL: HELPS  TO  GET  HELP  FROM  TEACHER  ON  MATH 

5206705  FEEL: LIKE  GETTING  HELP  FROM  FRIEND  ON  MATH 

5206706  FEEL: HELPS  TO  GET  HELP  FROM  FRIEND  ON  MATH 


M4 
M4 
M7 
M7 
M7 
M7 


2 
3 
3 
4 
5 
6 
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Table  C.5 
(continued) 


BLOCK  ITEM 

MWlEx  -  Computer  usage 

B003801      HOW  MANY  HOURS  DO  YOU  SPEND  ON  HOMEWORK  Bl  14 

S205601      HOW  MUCH  TIME  SPKNT/WK  ON  MATH  HOMEWORK  M4  5 

S 207608      HOW  OFTEN  DO  YOU  DO  MATH  HOMEWO?*  M6  8 

B004301      HAVE  YOU  USED  COMPUTER  W -'KEYBOARD  AND  SCREEN  Bl  19 

B004401      DOES  FAMILY  OWN  COMPUTER  W/KEYBOARP  .\ND  SCREEN  Bl  20 

B004501      ARE  YOU  STUDYING  COMPUTERS  Bl  21 

B004601      DO  YOU  USE  COMPUTERS  FOR  MATH,  READING,  ETC  Bl  22 
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Table  C.6 


Mathematics  WARM  Variables 
Grade  7/Age  13 


MW2AX  -  Positive  attitude 

S202201  FEEL:WILLING  TO  WORK  HARD  TO  DC  WELL  IN  MATH 

S202204  FEEL: MATH  USEBTJL  IN  SOLVING  EVERY  DAY  PROBLEMS 

S202206  FZEL: ENJOY  MATHEMATICS 

5202901  FEEL: I  AM  GOOD  AT  MATHEMATICS 

5202902  FEEL:MATH  HELPS  A  PERSON  THINK  LOGICALLY 
S202905  FEEL: I  AM  TAKING  MATH  ONLY  BECAUSE  I  HAVE  TO 
S202908  FEEL: I  WOULD  LIKE  TO  TAKE  MORE  MATHEMATICS 
S206001  DO  YOU  FEEL  AS  GOOD  IN  MATH  AS  OTHERS  IN  CLASS 
S209001  WILL  YOUR  CAREER  REQUIRE  USE  OF  MATH  SKILLS 
S209501  FEEL  MOST  OF  MATH  HAS  PRACTICAL  USE 

5209509  AGREE  WANT  PERSONAL  SUCCESS  IN  MATH 

5209510  FEEL  PARENTS  WANT  ME  TO  DO  WELL  IN  MATH 

5209511  FEEL  GOOD  WHEN  SOLVE  A  MATH  PROBLEM  ALONE 
S209515  AGREE  LIKES  TO  BE  CHALLENGED  W/DIFFICULT  PROBLEM 
S2 11003  DO  YOU  LIKE  OR  DISLIKE  MATHEMATICS 

S211008  HOW  EASY  OR  HARD  IS  MATHEMATICS 

S211013  HOW  IMPORTANT  OR  NOT  IS  MATHEMATICS 

S211305  DO  YOU  AGREE  A  GOOD  GRADE  IN  MATH  IS  IMPORTANT 


BLOCK  ITEM 


M2 
M2 
M2 
M3 
M3 
M3 
M3 
M5 
M5 
M7 
M7 
M7 
M7 
M7 
M8 
MS 
MS 
M9 


1 
4 
6 
1 
2 
5 
S 
16 
17 
1 
9 
10 
11 
15 
3 
S 
13 
5 


MW2Bx  -  Seeks  assistance 


5207501  HOW  OFTEN  DO  YOU  LISTEN  TO  MATH  LESSON  EXPLAINED  M6  1 

5207502  HOW  OFTEN  WATCH  TEACHER  WORK  PROBLEM  ON  BOARD  M6  2 

5207503  HOW  OFTEN  USE  A  MATH  TEXTBOOK  M6  3 
S207507  HOW  OFTEN  WORKS  MATH  PROBLEMS  ALONE  M6  7 
S 2 07 5 10  HOW  OFTEN  DO  YOU  TALK  ABOUT  MATH  IN  CLASS  M6  10 
S 207 5 12  HOW  OFTEN  DO  YOU  DO  MATH  HOMEWORK  M6  12 

5211401  HOW  OFTEN  GETS  INDIVIDUAL  HELP  FROM  MA  TEACHER  M9  9 

5211402  HOW  OFTEN  GETS  HELP  FROM  CLASSMATE  W/MATH  M9  10 

5211403  HOW  OFTEN  HELPS  CLASSMATE  WITH  MATH  M9  11 


MW2Cx  -  Does  math  on  own 


5207508  HOW  OFTEN  DO  YOU  DO  MATH  LABORATORY  ACTIVITIES 

5207509  HOW  OFTEN  DO  YOU  DO  MATH  REPORTS  AND  PROJECTS 

5211404  HOW  OFTEN  CHOOSE  MATH  TOPICS  YOU  WANT  TO  STUDY 

5 2 1140 5  HOW  OFTEN  PLAYS  MATH  GAMES 

5211406  HOW  OFTEN  WORKS  AHEAD  IN  MATH  BOOK 

5211407  HOW  OFTEN  DOES  MATH  PROBLEMS  NOT  ASSIGNED 

5211408  HOW  OFTEN  STUDY  MATH  TOPICS  NOT  IN  TEXTBOOK 


M6 
M6 
M9 
M9 
M9 
M9 
M9 


8 
9 
12 
13 
14 
15 
16 
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Table  C.6 
(continued) 


BLOCK  ITEM 


MW2Dx  -  Computer  usage 


B004301 

HAVE  YOU  USED  COMPUTER  W/KEYBOARD  AND  SCREEN 

Bl 

18 

UObb  FAMILY  OWN  COMPUTER  W/KEYBOARD  AND  SCREEN 

Bl 

19 

B004501 

ARE  YOU  STUDYING  COMPUTERS 

Bl 

20 

B004601 

DO  YOU  USE  COMPUTERS  FOR  MATH,  READING,  ETC 

Bl 

21 

S201601 

EVER  STUDY  MATH  THROUGH  COMPUTER  INSTRUCTION 

Ml 

1 

o^U  LOU  J 

hVhK  USh  COMPUTER  TO  SOLVE  LINEAR  PROGRAM  PROBLEM 

Ml 

5 

<;9ni  AHA 

hvhK  USh  COMPUTER  TO  SOLVE  A  MATH  PROBLEM 

Ml 

6 

o^U  XDUO 

hVhR  USE  COMPUTER  TO  PROCESS  BUSINESS , SCI , SOC  INFO 

Ml 

8 

J  xo 

hvhK  USh  COMPUTER  TO  PERFORM  STAT  ANALYSIS 

Ml 

9 

O XO  LKJ 

hVhK  WKiih  FROGKAM  TO  SOLVE  LINEAR  PROGRAM  PROBLEM 

Ml 

10 

D  ^     X  LI  X  J. 

hvhk  WKliii  rKUbKArl  iO  SOLVE  A  MATH  PROBLEM 

Ml 

11 

S20161 9 

k-J  ^  W  JL  \^  JL  ^ 

"P\7"PR   UPTTF   PPnPPAM"  Tn   pt  av   a       a mrt? 

Ml 

12 

S201613 

PVFR  URTTT?  PPfiPPAM  '^n  PPnr'TTCC   dttotmuoc   or^-x   nr\j^  txit* 
tviLts.  wK.xiii  rKUUKAXT.  i.u  rKUUlLob  UUbiNhbb  ,  SCI ,  SOC ,  INF 

M" 

13 

S201614 

PUFR   URTTT?   PPHPPAM   Tn   PUPtmPM"   otat  axtatxtcto 
E^vLiis,  vits.xL£4  rts.UKjtsj\sri  iu  rhKrUKW.  biAi  ANALYSIS 

Ml 

14 

S206101 

HOW  OFTEN  USE  A  CALCULATOR  IN  MATH  CLASS 

M5 

1 

S206102 

HOW  OFTEN  USE  A  CALCULATOR  IN  SCIENCE  CLASS 

M5 

2 

S206103 

HOW  OFTFN  TT^F  A    TAT  TTTT  ATOP    TM  nTUTTP   r-T  aoot70 

M5 

3 

S206104 

HOW  OFTFN  TT^F   A    TAT  TTTT  ATnP    nTTTCTnT?  m?  cnunr^j 
rn^H   yjr  LiLVi   uoiL  /i  L»AX,L»UXjA.iUK  UUioXUci  Ur  SOrlUuL 

M5 

4 

S208701 

HOW  OFTFN  no  YOTT  TT<3F  A  rATrTTTATnP 

M4 

14 

S208801 

nSF  A  HAT  flTTTATOR   TN  MATT4  TTnp  unMtriTnPV 

M5 

5 

S208802 

^  \y  w  \J  \J 

TTCF    A    PAT  PTTT  ATOP    TM  MATU   T?nP    r'Utrr'i/T \ir^  AXTOTTr<no 
uoiL  A  uAXiUUXAXUK.  xiN  rLAin  rUK  UniLUKiNG  ANSWERS 

M5 

6 

S208803 

USE  A  CAT  CUTJ^TOR   TN  MATH  FOP  POTTTTNF  rnMPTTTATTnMC 

Md 

7 

S208804 

USE  A  CALCULATOR  IN  MATH  FOR  SOLVING  PROBLEMS 

M5 

8 

S208805 

USE  A  CALCULATOR  IN  MATH  FOR  TAKING  TF  JTS 

M5 

9 

S208806 

USE  A  CALCULATOR  IN  MATH  FOR  SOMEl.    ^JG  ELSE 

M5 

10 

MW2Fx  -  Encouragement  received 

S205401 

WHAT  EXTENT  HAVE  PARENTS  ENCOURAGED  MA  COURSES 

M4 

1 

2205402 

WHAT  EXTENT  HAVE  SIBLINGS  ENCOURAGED  MA  COURSES 

M4 

2 

S205403 

WHAT  EXTENT  HAVE  RELATIVE  ENCOURAGED  MA  COURSES 

M4 

3 

S2C5404 

WHAT  EXTENT  HAVE  TEACHERS  ENCOURAGED  MA  COURSES 

M4 

4 

S205405 

WHAT  EXTENT  HAVE  COUNSELOR  ENCOURAGED  MA  COURSES 

M4 

5 

S205406 

WHAT  EXTENT  HAVE  PEERS  ENCOURAGED  MA  COURSES 

M4 

6 

-535- 


ERIC 


583 


Table  C.7 


Mathematics  WARM  Variables 
Grade  11/Age  17 


MW3Ax  -  Positive  attitude 

5202901  FEEL: I  AM  GOOD  AT  MATHEMATICS 

5202902  FEEL: MATH  HELPS  A  PERSON  THINK  LOGICALLY 
S202905  FEEL: I  AM  TAKING  MATH  ONLY  BECAUSE  I  HAVE  TO 
S202908  FEEL: I  WOULD  LIKE  TO  TAKE  MORE  MATHEMATICS 
S202911  FEEL: I  UNDERSTAND  WHAT  WE  TALK  ABOUT  IN  MATH 
S206001  DO  YOU  FEEL  AS  GOOD  IN  MATH  AS  OTHERS  IN  CLASS 
S209001  WILL  YOUR  CAREER  REQUIRE  USE  OF  MATH  SKILLS 
S209501  FEEL  MOST  OF  MATH  HAS  PRACTICAL  USE 

5209509  AGREE  WANT  PERSONAL  SUCCESS  IN  MATH 

5209510  FEEL  PARENTS  WANT  ME  TO  DO  WELL  IN  MATH 

5209511  FEEL  GOOD  WHEN  SOLVE  A  MATH  PROBLEM  ALONE 
S209515  AGREE  LIKES  TO  BE  CHALLENGED  W/DIFFICULT  PROBLEM 
S211003  DO  YOU  LIKE  OR  DISLIKE  MATHEMATICS 

S211008  HOW  EASY  OR  HARD  IS  MATHEMATICS 

S211013  HOW  IMPORTANT  OR  NOT  IS  MATHEMATICS 

S212001  FEEL:WILLING  TO  WORK  HARD  TO  DO  WELL  IN  MATH 

S212004  FEEL:MATH  USEFUL  IN  SOLVING  EVERYDAY  PROBLEMS 

S212006  FEEL: ENJOY  MATHEMATICS 

S212008  FEEL:A  GOOD  GRADE  IN  MATH  IS  IMPORTANT  TO  ME 

5212101  FEEL: I  UNDERSTAND  WHAT  WE  TALK  ABOUT  IN  MATH 

5212102  FEEL: GOOD  AT  WORKING  WITH  NUMBERS 

5212103  FEEL: DOING  MATHEMATICS  MAKES  ME  NERVOUS 

5212104  FEEL: MATHEMATICS  IS  BORING  FOR  ME 

5212105  FEEL:WILLING  TO  WORK  HARD  TO  DO  WELL  IN  MATH 


BLOCK  ITEM 


M3 
M3 
M3 
M3 
M3 
M5 
M5 
M7 
M7 
M7 
M7 
M7 
M8 
M8 
M8 
M9 
M9 
M9 
M9 
M9 
M9 
M9 
M9 
M9 


1 
2 
5 
8 
11 
16 
17 
1 
9 
10 
11 
15 
3 
8 
13 
1 
4 
6 
8 
12 
13 
14 
15 
16 


MW3Bx  -  Seeks  assistance 


5203701  HOW  OFTEN  DID  YOU  TAKE  MATH  TESTS  M2  1 

5203702  HOW  OFTEN  DID  YOU  DO  MATH  HOMEWORK  M2  2 

5203703  HOW  OFTEN  DID  YOU  HELP  A  CLASSMATE  DO  MATH  M2  3 

5203705  HOW  OFTEN  DID  YOU  LISTEN  TO  LESSON  EXPLANATION  M2  5 

5203706  HOW  OFTEN  DID  YOU  SEE  TEACHER  DO  MATH  ON  BOARD  M2  6 

5203707  HOW  OFTEN  DID  YOU  GET  TEACHER'S  HELP  WITH  MATH  M2  7 
S203711  HOW  OFTEN  DID  YOU  GET  FRIEND'S  HELP  WITH  MATH  M2  11 

5203713  HOW  OFTEN  DID  YOU  DISCUSS  MATHEMATICS  IN  CLASS  M2  13 

5203714  HOW  OFTEN  DID  YOU  WORK  MATH  PROBLEMS  AT  BOARD  M2  14 
S206601  HOW  MUCH  TIME  SPENT/WK  ON  MATH  HOMEWORK  M4  11 

5207501  HOW  OFTEN  DO  YOU  LISTEN  TO  MATH  LESSON  EXPLAINED  M6  1 

5207502  HOW  OFTEN  WATCH  TEACHER  WORK  PROBLEM  ON  BOARD  M6  2 

5207503  HOW  OFTEN  USE  A  MATH  TEXTBOOK  M6  3 
S207507  HOW  OFTEN  WORKS  MATH  PROBLEMS  ALONE  M6  7 
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Table  C.7 
(continued) 


(continued)  MW3Bx  -  Seeks  assistance 


BLOCK  ITEM 


S207510 

HOW 

OFTEN 

S211401 

HOW 

OFTEN 

S211402 

HOW 

OFTEN 

S211403 

HOW 

OFTEN 

S211410 

HOW 

OFTEN 

MW3Cx  -  D< 

S203704 

HOW 

OFTEN 

S203708 

HOW 

OFTEN 

S203709 

HOW 

OFTEN 

S203710 

HOW 

OFTEN 

S203712 

HOW 

OFTEN 

S207508 

HOW 

OFTEN 

S207509 

HOW 

OFTEN 

S211404 

HOW 

OFTEN 

S211405 

HOW 

OFTEN 

S211406 

HOW 

OFTEN 

S211407 

HOW 

OFTEN 

S211408 

HOW 

OFTEN 

Does  math  on  own 

DID  YOU  PLAY  MATHEMATICS  GAMES 
DID  YOU  MAKE  REPORTS/PROJECTS  IN  MATH 
DID  YOU  WORK  AHEAD  IN  YOUR  MATH  BOOK 
DID  YOU  DO  MATH  PROBLEMS  NOT  ASSIGNED 
DID  YOU  STUDY  TOPIC  NOT  IN  TEXTBOOK 
DO  YOU  DO  MATH  LABORATORY  ACTIVITIES 
DO  YOU  DO  MATH  REPORTS  AND  PROJECTS 
CHOOSE  MATH  TOPICS  YOU  WANT  TO  STUDY 
PLAYS  MATH  GAMES 
WORKS  AHEAD  IN  MATH  BOOK 
DOES  MATH  PROBLEMS  NOT  ASSIGNED 
STUDY  MATH  TOPICS  NOT  IN  TEXTBOOK 


M6 

MIO 

MIO 

MIO 

MIO 


M2 

M2 

M2 

M2 

M2 

M6 

M6 

MIO 

MIO 

MIO 

MIO 

MIO 


10 
3 
4 
5 
2 


4 
8 
9 
10 
12 
8 
9 
6 
7 
8 
9 
10 


MW3Dx  -  Computer  usage 

B004301  HAVE  YOU  USED  COMPUTER  W/KEYBOARD  AND  SCREEN  Bl  18 

B004401  DOES  FAMILY  OWN  COMPUTER  W/KEYBOARD  AND  SCREEN  Bl  19 

S20i601  EVER  STUDY  MATH  THROUGH  COMPUTER  INSTRUCTION  Ml  1 

5201605  EVER  USE  COMPUTER  TO  SOLVE  LINEAR  PROGRAM  PROBLEM  Ml  5 

5201606  EVER  USE  COMPUTER  TO  SOLVE  A  MATH  PROBLEM  Ml  6 
S201608  EVER  USE  COMPUTER  TO  PROCESS  BUSINESS , SCI , SOC  INFO  Ml  8 
S201612  EVER  WRITE  PROGRAM  TO  PLAY  A  GAME  Ml  12 
S20iei3  EVER  WRITE  PROGRAM  TO  PROCESS  BUSINESS , SCI , SOC , INF  Ml  13 
S201614  EVER  WRITE  PROGRAM  TO  PERFORM  STAT  ANALYSIS  Ml  14 
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Table  C.7 
(continued) 


BLOCK  ITEM 


S206101 
S206102 
S206103 
S206104 
S208701 
S208801 
S208802 
S208803 
S208804 
S208805 
S208806 


MW3Ex  -  Calculator  usage 

HOW  OFTEN  USE  A  CALCULATOR  IN  MATH  CLASS 
HOW  OFTEN  USE  A  CALCULATOR  IN  SCIENCE  CLASS 
HOW  OFTEN  USE  A  CALCULATOR  IN  OTHER  CLASSES 
HOW  OFTEN  USE  A  CALCULATOR  OUTSIDE  OF  SCHOOL 
HOW  OFTEN  DO  YOU  USE  A  CALCULATOR 

CALCULATOR  IN  MATH  FOR  HOMEWORK 
CALCULATOR  IN  MATH 


USE 
USE 
USE 
USE 
USE 
USE 


FOR  CHECKING  AJJSWERS 
CALCULATOR  IN  MATH  FOR  ROUTINE  COMPUTATIONS 
CALCULATOR  IN  MATH  FOR  SOLVING  PROBLEMS 
CALCULATOR  IN  MATH  FOR  TAKING  TESTS 
CALCULATOR  IN  MATH  FOR  SOMETHING  ELSE 


M5 
M5 
M5 
M5 
M4 
M5 
M5 
M5 
M5 
M5 
M5 


1 
2 
3 
4 

14 
5 
6 
7 
8 
9 

10 


MW3Fx  -  Encouragement  received 


5205401  WHAT  EXTENT  HAVE  PARENTS  ENCOURAGED  MA  COURSES  M4  1 

5205402  WHAT  EXTENT  HAVE  SIBLINGS  ENCOURAGED  MA  COURSES  M4  2 

5205403  WHAT  EXTENT  HAVE  RELATIVE  ENCOURAGED  MA  COURSES  M4  3 

5205404  WHAT  EXTENT  HAVE  TEACHERS  ENCOURAGED  MA  COURSES  M4  4 

5205405  WHAT  EXTENT  HAVE  COUNSELOR  ENCOURAGED  MA  COURSES  M4  5 

5205406  WHAT  EXTENT  HAVE  PEERS  ENCOUPAGED  MA  COURSES  M4  6 


-538- 
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Table  C,8 

Derivation  of  Mathematics  WARM  Scores 


COHORT 


RESPONSES 


1 

2 

3 

1 

2 

3  4 

1 

B003801 

E 

1 

2 

3  4 

2 

B004301 

F 

D 

D 

5 

1 

3 

B004401 

F 

D 

D 

5 

1 

4 

B004501 

F 

D 

5 

1 

5 

B004601 

F 

D 

5 

1^ 

6 

B005312 

D 

5 

1 

7 

B005313 

D 

5 

1 

8 

S201601 

D 

D 

5 

1 

1 

9 

S201605 

D 

D 

5 

1 

1 

10 

S201606 

D 

D 

5 

1 

1 

11 

S201608 

D 

D 

5 

1 

1 

12 

S201609 

D 

D 

5 

1 

1 

13 

S201610 

D 

D 

5 

1 

1 

14 

S201611 

D 

D 

5 

1 

1 

15 

S201612 

D 

D 

5 

1 

1 

16 

S201613 

D 

D 

5 

1 

1 

17 

S201614 

D 

D 

5 

1 

1 

18 

S202201 

A 

1 

2 

3  4 

19 

S202204 

A 

1 

2 

3  4 

20 

S202206 

A 

1 

2 

3  4 

21 

S202901 

A 

A 

1 

2 

3  4 

22 

S202902 

A 

A 

1 

2 

3  4 

23 

S202905 

A 

A 

5 

4 

3  2 

24 

S202908 

A 

A 

1 

2 

3  4 

25 

S202911 

A 

1 

2 

3  4 

26 

S203701 

B 

5 

3 

1 

27 

S203702 

B 

5 

3 

1 

28 

S203703 

B 

5 

3 

1 

29 

S203704 

C 

5 

3 

1 

30 

S203705 

B 

5 

3 

1 

31 

S203706 

B 

5 

3 

1 

32 

S203707 

B 

5 

3 

1 

33 

S203708 

C 

5 

3 

1 

34 

S203709 

C 

5 

3 

1 

35 

S203710 

C 

5 

3 

1 

36 

S203711 

B 

5 

3 

1 

37 

S203712 

C 

5 

3 

1 

38 

S203713 

B 

5 

3 

1 

39 

S203714 

B 

5 

3 

1 

40 

S205401 

F 

F 

5 

3 

1 

5 
5 
5 
5 
5 
1 
5 
5 


8  MISS 

M 
M 
M 
M 
M 
M 
M 


M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 
M 


FRir 
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Table  C.S 
(continued) 


COHORT   RESPONSES 


1 

2 

3 

1 

2 

3 

4 

5 

6 

1 

8  MISS 

41 

S205402 

F 

F 

5 

3 

1 

M 

42 
43 

S205403 
S205404 

F 
F 

F 
F 

5 
5 

3 
3 

1 
1 

M 
M 

44 

S205405 

F 

F 

_/ 

3 

1 

M 

45 

S205406 

F 

F 

5 

3 

1 

M 

46 

S205502 

D 

5 

1 

1 

47 

S205503 

D 

5 

1 

1 

48 

S205601 

E 

1 

2.33 

3.67 

5 

M 

M 

49 

S205701 

A 

5 

1 

M 

M 

50 

S205901 

A 

5 

3 

1 

M 

51 

S205902 

A 

5 

3 

1 

M 

52 

S205903 

A 

1 

3 

5 

M 

53 

S205904 

A 

1 

3 

5 

M 

54 

S205905 

A 

5 

3 

1 

M 

55 

S205906 

A 

5 

3 

1 

M 

56 

S206001 

A 

A 

5 

1 

M 

57 

S206101 

B 

E 

E 

5 

4 

3 

2 

1 

M 

58 

S206102 

B 

E 

E 

5 

4 

3 

2 

1 

M 

59 

S206103 

B 

E 

E 

5 

4 

3 

2 

1 

M 

60 

S206104 

B 

E 

E 

5 

4 

3 

2 

1 

M 

61 

S206601 

B 

1 

2 

3 

4 

5 

1 

62 

S206701 

A 

5 

3 

1 

M 

63 

S206702 

A 

5 

3 

1 

M 

64 

S206703 

D 

5 

3 

1 

M 

65 

S206704 

D 

5 

3 

1 

M 

66 

S206705 

D 

5 

3 

1 

M 

67 

S206706 

D 

5 

3 

1 

M 

68 

S207101 

A 

1 

3 

5 

M 

69 

S207501 

B 

B 

5 

4 

3 

2 

1 

M 

70 

S207502 

B 

B 

5 

4 

3 

2 

1 

M 

71 

S207503 

B 

B 

5 

4 

3 

2 

1 

M 

72 

S207507 

B 

B 

5 

4 

3 

2 

1 

M 

73 

S207508 

C 

5 

4 

3 

2 

1 

M 

74 

S207509 

C 

C 

5 

4 

3 

2 

1 

M 

75 

S207510 

B 

B 

5 

4 

3 

2 

1 

M 

76 

S207512 

B 

5 

4 

3 

2 

1 

M 

77 

S207601 

C 

5 

4 

3 

2 

1 

M 

78 

S207602 

C 

5 

4 

3 

2 

1 

M 

79 

S207603 

c 

5 

4 

3 

2 

1 

M 

80 

S207604 

c 

5 

4 

3 

2 

1 

M 
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Table  C.8 
(continued) 


COHORT 


1 

2 

3 

81 

S207605 

C 

82 

S207608 

E 

83 

S208701 

E 

84 

S208801 

E 

E 

85 

S208802 

E 

E 

86 

S208803 

E 

E 

87 

S208804 

E 

E 

88 

S208805 

E 

E 

89 

S208806 

E 

E 

90 

S209001 

A 

A 

91 

S209501 

A 

A 

92 

S209509 

A 

A 

93 

S209510 

A 

A 

94 

S209511 

A 

A 

95 

S209515 

A 

A 

96 

S211003 

A 

A 

97 

S211008 

A 

A 

98 

S211013 

A 

A 

99 

S211305 

A 

100 

S211401 

B 

B 

101 

S211402 

B 

B 

102 

S211403 

B 

B 

103 

S211404 

C 

C 

104 

S211405 

C 

C 

105 

S211406 

C 

C 

106 

S211407 

C 

C 

107 

S211408 

C 

C 

108 

S211410 

B 

109 

S212001 

A 

110 

S212004 

A 

111 

S212006 

A 

112 

S212008 

A 

113 

S212101 

A 

114 

S212102 

A 

115 

S212103 

A 

116 

S212104 

A 

117 

S212105 

A 

ERIC 


RESPONSES 


12345678  MISS 


5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 
1 1 

5 

4 

3 

2 

1 

M 
1 1 

5 

1 

1 
J. 

5 

1 

1 
J. 

5 

1 

1 

5 

1 

5 

1 

1 

5 

1 

1 

5 

1 

3 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

1 

2 

3 

4 

5 

M 

M 

5 

4 

3 

2 

1 

M 

M 

1 

2 

3 

4 

5 

M 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

5 

3 

1 

M 

5 

3 

1 

M 

1 

3 

5 

M 

1 

3 

5 

M 

5 

3 

1 

M 

Table  C.9 
Science  WARM  Variables 
Grade  3/Age  9 


BLOCK  ITEM 


SWlAx  -  Use  of  scientific  apparatus 


5400101  HAVE  YOU  EVER  USED 

5400102  HAVE  YOU  EVER  USED 

5400103  HAVE  YOU  EVER  USED 
S400105  HAVE  YOU  EVER  USED 
S400L09  HAVE  YOU  EVER  USED  A  YARDSTICK 
S400110  HAVE  YOU  EVER  USED  A  CALCULATOR 


A  METER  STICK 
A  SCALE  TO  WEIGH  THINGS 
A  MAGNIFYING  GIASS 
A  THERMOMETER 


S3 
S3 
S3 
S3 
S3 
S3 


1 

2 
3 
5 
9 
10 


SWlBx  -  Home  encouragement  &  support 


5401601  AT  HOME  DO  YOU  TALK  ABOUT  Wl^AT  YOU  LEARNED  S4  2 

5401602  AT  HOME  DO  YOU  GET  HELP  W/SCIENCE  HOMEWORK  S4  3 

5401603  AT  HOME  DO  YOU  GET  HELP  W/SCIENCE  PROJECTS  S4  4 

5401604  AT  HOME  DOES  SOMEONE  SHOW  YOU  HOW  DO  SCIENCE  EXP  S4  5 

5401605  AT  HOME  DO  YOU  WATCH  &  DISCUSS  SCIENCE  PROGRAM  S4  6 

5401607  DO  YOU  TALK  ABOUT  SCIENCE  TOPICS  W/SOMEONE  AT  HOME  S4  8 

5401608  DO  YOU  READ  SCIENCE  BOOKS  W/SOMEONE  AT  HOME  S4  9 
S403201  HOW  MUCH  TIME  SPENT  ON  SCIENCE  HOMEWORK  EACH  WEEK  S6  1 


SWlCx  -  Doing  science  using  apparatus 


S400104  HAVE  YOU  EVER  USED  A  TELESCOPE 

S400106  HAVE  YOU  EVER  USED  A  MICROSCOPE 

S400108  HAVE  YOU  EVER  USED  A  STOPWATCH 

5400301  HAVE  YOU  WORKED/EXPERIMENTED  W/SHADOWS 

5400302  HAVE  YOU  WORKED/EXPERIMENTED  W/LIVING  PLANTS 

5400303  HAVE  YOU  WORKED/EXPERIMENTED  W/LIVING  ANIMALS 

5400304  HAVE  YOU  WORKED/EXPERIMENTED  W/BATTERY  &  BULBS 
S401606  DO  YOU  GO  TO  SCIENCE  MUSEUM  W/SOMEONE  FROM  HOME 


S3 
S3 
S3 
SI 
SI 
SI 
SI 
S4 


4 
6 
8 
1 
2 
3 
4 
7 


SWlDx  -  Value  placed  on  science 

S402101      WHEN  YOU  HAVE  SCIENCE  IN  SCHOOL  DO  YOU  LIKE  IT  S4  1 

S404301      WILL  KNOWING  A  LOT  ABOUT  SCIENCE  HELP  WHEN  GROWN  UP        S7  7 


ERIC 
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Table  C.9 
(continued) 


BLOCK  ITEM 

SWlEx  -  Science  classroom  activities 

S401501  HOW  OFTEN  DO  YOU  HAVE  SCIENCE  LESSON  IN  SCHOOL 

S401801  HOW  MANY  SCIENCE  EXP  DID  YOU  HAVE  LAST  MONTH 

S403801  HOW  OFTEN  DO  YOU  DO  SCIENCE  EXPERIMENTS 

S403901  HOW  OFTEN  DO  YOU  READ  SCIENCE  TEXTBOOK  IN  CLASS 


S5  1 

55  2 

56  2 
S6  3 


-5A3-  593 


Table  C.IO 
Science  WARM  Variables 
Grade  7/Age  13 


SW2AX  -  Value  placed  on  science 


BLOCK  ITEM 


5400701  AGREE/DISAGREE- SCI  HELPS  ME  UNDERSTAND  MY  BODY  S2 

5400702  AGREE/DISAGREE- SCIENCE  NOT  USEFUL  OUT  OF  CLASS  S2 
S401301  SCIENCE  CLASSES  ARE  USEFUL  S3 

5401303  SCIENCE  CLASSES  SHOULD  ^E  REQUIRED  IN  SCHOOL  S3 

5401304  SCIENCE  CLASSES  ARE  USEFUL  IN  EVERYDAY  LIFE  S3 

5401305  SCIENCE  CLASSES  WILL  BE  USEFUL  IN  THE  FUTURE  S3 
S402101  WHEN  YOU  HAVE  SCIENCE  IN  SCHOOL  DO  YOU  LIKE  IT  S4 
S402401  WILL  YOU  WORK  IN  AREA  THAT  REQ  SCIENCE  KNOWLEDGE  S7 
S402b01  HOW  OFTEN  IS  SCIENCE  CLASS  BORING  S7 

5402502  HOW  OFTEN  IS  SCIENCE  CLASS  FUN  S7 

5402503  HOW  OFTEN  DO  YOU  LOOK  FORWARD  TO  SCIENCE  CLASS  S7 

5402701  FEEL: IMPORTANT  TO  KNOW  SCIENCE  TO  GET  GOOD  JOB  S8 

5402702  FEEL:WILL  USE  SCIENCE  IN  MANY  WAYS  WHEN  AN  ADULT  S8 

5402703  FEEL: KNOWING  SCIENCE  WILL  HELP  ME  EARN  A  LIVING  S8 

5402704  FEEL: I  DON'T  EXPECT  TO  USE  SCIENCE  WHEN  I  GET  OUT  S8 

5402705  FEEL:  SCIENCE  WILL  BE  IMPORTANT  TO  ME  IN  MY  LIFE  S8 


1 

2 
1 
3 
4 
5 
1 
1 
6 
7 
8 
2 
3 
4 
5 
6 


SW2Bx  -  Science  classroom  acti:/ities 


5402001  HOW  OFTEN  SCIENCE  TEACHER  LECTURE 

5402002  HOW  OFTEN  DOES  SCI  TEACHER  DEMONSTRATE  PRINCIPLE 

5402003  HOW  OFTEN  DOES  SCI  TEACHER  ASK  FOR  REASONS  FOR  EXP 

5402004  HOW  OFTEN  DOES  SCI  TEACHER  ASK  YOU  TO  HYPOTHESIZE 

5402005  HOW  OFTEN  DOES  SCI  TEACHER  ASK  YOU- INTERPRET  DATA 

5402301  HOW  OFTEN  DO  YOU  SOLVE  SCIENCE  PROBLEMS 

5402302  HOW  OFTEN  DO  YOU  DO  EXPERIMENTS  BY  YOURSELF 

5402303  HOW  OFTEN  DO  YOU  DO  EXPERIMENTS  W/OTHER  STUDENTS 

5402304  HOW  OFTEN  DO  YOU  WRITE  UP  EXPERIMENTS 

5402306  HOW  OFTEN  DO  YOU  READ  ARTICLES  ABOUT  SCIENCE 

5402307  HOW  OFTEN  DO  YOU  DO  AN  ORAL  OR  WRITTEN  REPORT 


S9 

1 

S9 

2 

S9 

3 

S9 

4 

S9 

5 

S6 

2 

S6 

3 

S6 

4 

S6 

5 

S6 

7 

S6 

8 

-544- 


534 


Table  C.IO 
(continued) 


BLOCK  ITEM 

SW2Cx  -  Doing  science  &  home  support 

5400901  HOW  OFTEN  HAVE  YOU  HELPED  W/LITTER  CLEAN-UP  SI  1 

5400902  HOW  OFTEN  HAVE  YOU  SEPARATED  TRASH  FOR  RECYCLE  SI  2 

5401604  AT  HOME  DOES  SOMEONE  SHOW  YOU  HOW  DO  SCIENCE  EXP  S4  5 

5401605  AT  HOME  DO  YOU  WATCH  &  DISCUSS  SCIENCE  PROGRAM  S4  6 

5401607  DO  YOU  TALK  ABOUT  SCIENCE  TOPICS  W/SOMEONE  AT  HOME  S4  8 

5401608  DO  YOU  READ  SCIENCE  BOOKS  W/SOMEONE  Ai'  HOME  54  9 
S402201      HOW  OFTEN  TRIED  TO  FIX  SOMETHING  ELECTRICAL  S7  2 

5402203  HOW  OFTEN  TRIED  TO  FIGURE  OUT  UNHEALTHY  PLANT  S7  4 

5402204  HOW  OFTEN  TRIED  TO  FIGURE  OUT  UNHEALTHY  ANIMAL  S7  5 


SW2DX  -  Use  of  scientific  apparatus 


S402202  HOW  OFTEN  TRIED  TO  FIX  SOMETHING  MECHANICAL 

5402802  HAVE  YOU  EVER  USED  A  TELESCOPE 

5402803  HAVE  YOU  EVER  USED  A  MICROSCOPE 

5402804  HAVE  YOU  EVER  USED  A  STOPWATCH 
S402807  HAVE  YOU  EVER  USED  A  STETHOSCOPE 


S7 
S8 
S8 
S8 
S8 


3 
8 
9 
10 
13 


SW2Ex  -  Application  of  science  I 

5401201  CAN  SCIENCE  HELP  PREVENT  WORLDWIDE  STARVATION  SI  3 

5401202  CAN  SCIENCE  HELP  SAVE  US  FROM  AN  ENERGY  SHORTAGE  SI  4 

5401203  CAN  SCIENCE  HELP  FIND  CURES  FOR  DISEASES  SI  5 

5401206  CAN  SCIENCE  HELP  PREVENT  BIRTH  DEFECTS  SI  8 

5401207  CAN  SCIENCE  HELP  SAVE  OUR  NATURAL  RESOURCES  SI  9 

5401208  CAN  SCIENCE  HELP  REDUCE  AIR  &  WATER  POLLUTION  SI  10 

5401209  CAN  SCIENCE  HELP  REDUCE  WORLD  OVERPOPULATION  Si  11 


SW2FX  -  Solving  world  problems 


5400801  CAN  YOU  HELP  SOLVE  POLLUTION 

5400802  CAN  YOU  HELP  SOLVE  ENERGY  WASTE 

5400803  CAN  YOU  HELP  SOLVE  FOOD  SHORTAGES 

5400804  CAN  YOU  HELP  SOLVE  OVERPOPULATION 

5400805  CAN  YOU  HELP  SOLVE  NATURAL  RESOURCE  SUPPLY 

5400806  CAN  YOU  HELP  SOLVE  ACCIDENTS 


S2 
S2 
S2 
S2 
S2 
S2 


4 
5 
6 
7 
8 
9 


•545- 
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Table  C.IO 
(continued) 


SW2Gx  -  Application  of  science  II 


BLOCK  ITEM 


5403101  WILL  SCIENTIFIC  RESEARCH  ELIMINATE  ACID  RAIN  S9  6 

5403102  WILL  SCIENTIFIC  RESEARCH  ELIMINATE  AIR  POLLUTION  S9  7 

5403103  WILL  SCIENTIFIC  RESEARCH  ELIMINATE  FACTORY  POLLUTI  S9  8 

5403104  WILL  SCIENTIFIC  RESEARCH  ELIMINATE  TOXIC  WASTE  S9  9 


-546- 


Table  C.ll 
Science  WARM  Variables 
Grade  11/Age  17 


SW3Ax  -  Value  placed  on  science 


BLOCK  ITEM 


5400702  AGREE/DISAGREE- SCIENCE  NOT  USEFUL  OUT  OF  CLASS  S2  2 

5400703  AGREE/DISAGREE- PROCEDURES  GOOD  ONLY  IN  A  LAB  S2  3 

5401301  SCIENCE  CLASSES  ARE  USEFUL  S3  1 

5401302  SCIENCE  CLASSES  ARE  IRRELEVANT  TO  REAL  WORLD  S3  2 

5401303  SCIENCE  CLASSES  SHOULD  BE  REQUIRED  IN  SCHOOL  S3  3 

5401304  SCIENCE  CLASSES  ARE  USEFUL  IN  EVRYDAY  LIFE  S3  4 

5401305  SCIENCE  CLASSES  WILL  BE  USEFUL  IN  THE  FUTURE  S3  5 
S401901  OUTSIDE  SCHOOL  READ  BOOKS /ARTICLES  ABOUT  SCIENCE  S7  1 
S402101  WHEN  YOU  HAVE  SCIENCE  IN  SCHOOL  DO  YOU  LIKE  IT  S4  1 
S402401  WILL  YOU  WORK  IN  AREA  THAT  REQ  SCIENCE  KNOWLEDGE  S4  11 

5402501  HOW  OFTEN  IS  SCIENCE  CLASS  BORING  SIO  12 

5402502  HOW  OFTEN  IS  SCIENCE  CLASS  FUN  SIO  13 

5402503  HOW  OFTEN  DO  YOU  LOOK  FORWARD  TO  SCIENCE  CLASS  SIO  14 

5402701  FEEL: IMPORTANT  TO  KNOW  SCIENCE  TO  GET  GOOD  JOB  Sll  5 

5402702  FEEL:WILL  USE  SCIENCE  IN  MANY  WAYS  WHEN  AN  ADULT  Sll  6 

5402703  FEELrKNOWING  SCIENCE  WILL  HELP  ME  EARN  A  LIVING  Sll  7 

5402704  FEEL: I  DON'T  EXPECT  TO  USE  SCIENCE  WHEN  I  GET  OUT  Sll  8 

5402705  FEEL: SCIENCE  WILL  BE  IMPORTANT  TO  ME  IN  MY  LIFE  Sll  9 


SW3Bx  -  Application  of  science  I 


5401201  CAN  SCIENCE  HELP 

5401202  CAN  SCIENCE  HELP 

5401206  CAN  SCIENCE  HELP 

5401207  CAN  SCIENCE  HELP 

5401208  CAN  SCIENCE  HELP 

5401209  CAN  SCIENCE  HELP 
S402902  SHOULD  SCIENTIST 
S402904  SHOULD  SCIENTIST 
S402906  SHOULD  SCIENTIST 
S402908  SHOULD  SCIENTIST 


PREVENT  WORLDWIDE  STARVATION  SI  3 

SAVE  US  FROM  AN  ENERGY  SHORTAGE  SI  4 

PREVENT  BIRTH  DEFECTS  SI  8 

SAVE  OUR  NATURAL  RESOURCES  SI  9 

REDUCE  AIR  &  WATER  POLLUTION  SI  10 

REDUCE  WORLD  OVERPOPULATION  SI  11 

RECEIVE  $  STUDY: AIR  POLLUTION  S9  2 

RECEIVE  $  STUDY: ENERGY  PROBLEM  S9  4 

RECEIVE  $  STUDY: POPULATION  PROBLE  S9  6 

RECEIVE  $  STUDY: SOLAR  ENERGY  S9  8 
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Table  C.ll 
(continued) 


BLOCK  ITEM 


SW3Cx  -  Science  &  math  background 

B005401  WHICH  DESCRIBES  YOUR  GRADES  SO  FAR 

NCOMP  NUMBER  OF  COMPUTER  COURSES  TAKEN 

NMATH  HIGHEST  LEVEL  OF  MATH  COURSE  TAKEN 

NSCI  HIGHEST  LEVEL  OF  SCIENCE  COURSE  TAKEN 

S403306  HOW  MUCH  HAVE  YOU  STUDIED : CHEMISTRY 

S403308  HOW  MUCH  HAVE  YOU  STUDIED : PHYSICS 

S403-^01  HAVE  YOU  TAKEN  MORE  SCIENCE  COURSES  THAN  REQUIRED 


Bl  39 

(ETS  COMPOSITE) 

(ETS  COMPOSITE) 

(ETS  COMPOSITE) 


SIO 
SIO 
SIO 


6 
8 
11 


SW3Dx  -  Science  interest  &  application 

5400801  CAN  YOU  HELP  SOLVE  POLLUTION  S2  4 

5400802  CAN  YOU  HELP  SOLVE  ENERGY  WASTE  S2  5 
S400805  CAN  YOU  HELP  SOLVE  NATURAL  RESOURCE  SUPPLY  32  8 
S401902  OUTSIDE  SCHOOL  READ  BOOKS  ABOUT  SCIENTISTS  37  2 
S401910  OUTSIDE  SCHOOL  TALK  ABOUT  SCIENCE  TOPIC  W/FRIENDS  37  10 
3401911  OUTSIDE  SCHOOL  LISTEN  TO  TALKS  ABOUT  SCIENCE  S7  11 
S402306  HOW  OFTEN  DO  YOU  READ  ARTICLES  ABOUT  SCIENCE  S6  7 


SW3EX 


Science  classroom  activities 


S402001 

HOW 

OFTEN 

SCIENCE  TEACHER  LECTURE 

S8 

1 

S402002 

HOW 

OFTEN 

DOES  SCI  TEACHER  DEMONSTRATE  PRINCIPLE 

S8 

2 

S402003 

HOW 

OFTEN 

DOES  SCI  TEACHER  ASK  FOR  REASONS  FOR  EXP 

S8 

3 

S402004 

HOW 

OFTEN 

DOES  SCI  TEACHER  ASK  YOU  TO  HYPOTHESIZE 

S8 

4 

S402005 

HOW 

OFTEN 

DOES  SCI  TEACHER  ASK  YOU- INTERPRET  DATA 

S8 

5 

S402301 

HOW 

OFTEN 

DO  YOU  SOLVE  SCIENCE  PROBLEMS 

S6 

2 

S402302 

HOW 

OFTEN 

DO  YOU  DO  EXPERIMENTS  BY  YOURSELF 

S6 

3 

S402303 

HOW  OFTEN  DO  YOU  DO  EXPERIMENTS  W/OTHER  STUDENTS 

S6 

4 

S402304 

HOW 

OFTEN 

DO  YOU  WRITE  UP  EXPERIMENTS 

S6 

5 

S402305 

HOW 

OFTEN 

DO  YOU  READ  YOUR  TEXTBOOK 

S6 

6 

S402307 

HOW 

OFTEN 

DO  YOU  DO  AN  ORAL  OR  WRITTEN  REPORT 

S6 

8 

SW3Fx  -  Science  background 

S401401  SHOULD  SCI  EXPERIMENT  ON  PEOPLE  W/0  APPROVAL 

S401404  SHOULD  SCI  CONTROL  THE  WAY  PEOPLE  ACT 

S402802  HAVE  YOU  EVER  USED  A  TELESCOPE 

S402804  HAVE  YOU  EVER  USED  A  STOPWATCH 

S402807  HAVE  YOU  EVER  USED  A  STETHOSCOPE 

5403301  HOW  MUCH  HAVE  YOU  STUDIED: GENERAL  SCIENCE 

5403302  HOW  MUCH  HAVE  YOU  STUDIED: LIFE  SCIENCE 


S3 

S3 

S9 

S9 

S9 

SIO 

SIO 


6 
9 
11 
13 
16 
1 
2 


ERIC 
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Table  C.ll 
(continued) 


SW3Gx  -  Hiscellaneous 


BLOCK  ITEM 


B003901  HOW  MUCH  TIME  EACH  DAY  IS  SPENT  ON  HOMEWORK 

S401602  AT  HOME  DO  YOU  GET  HELP  W/SCIENCE  HOMEWORK 

S401905  OUTSIDE  SCHOOL  FIX  SOMETHING  MECHANICAL 

5402201  HOW  OFTEN  TRIED  TO  FIX  SOMETHING  ELECTRICAL 

5402202  HOW  OFTEN  TRIED  TO  FIX  SOMETHING  MECHANICAL 
S402808  HAVE  YOU  EVER  USED  AN  ELECTRICITY  METER 

5403501  DO  YOU  USE  SCI  INFO  TO  DECIDE  WHAT  FOOD  TO  EAT 

5403502  DO  YOU  USE  SCI  INFO  TO  KEEP  HEALTHY 


Bl 
S4 

87 

Sll 

Sll 

89 

87 

87 


14 
4 

5 
1 
2 
17 
13 
14 


SW3Hx  -  Application  of  science  II 


5403101  WILL  SCIENTIFIC  RESEARCH  ELIMINATE 

5403102  WILL  SCIENTIFIC  RESEARCH  ELIMINATE 

5403103  WILL  SCIENTIFIC  RESEARCH  ELIMINATE 

8403 104  WILL  SCIENTIFIC  RESEARCH  ELIMINATE 

8403105  WILL  SCIENTIFIC  RESEARCH  ELIMINATE 

8403106  WILL  SCIENTIFIC  RESEARCH  ELIMINATE 


ACID  RAIN  88  6 

AIR  POLLUTION  88  7 

FACTORY  POLLUTI  88  8 

TOXIC  WASTE  88  9 

ENERGY  SUPPLY  SB  10 

LACK  OF  FOOD  88  11 
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Table  C,12 
Derivar.ion  of  Science  WARM  Scores 


COHORT 


RESPONSES 


1 

2 

3 

1 

2 

3 

4 

5 

6  7 

8 

MISS 

1 

B003901 

G 

M 

1 

2 

3 

4 

5 

M 

2 

B005401 

C 

5 

4. 

43 

3. 

86  3 

.29 

2.71 

2.14  1.57 

1 

M 

3 

NCOMP 

C 

1 

3 

5 

M 

4 

NMATH 

C 

1 

2 

3 

4 

5 

M 

M 

5 

NSCI 

C 

1 

2. 

33 

3. 

67 

5 

M 

M 

6 

S400101 

A 

5 

1 

1 

I 

7 

S400102 

A 

5 

1 

1 

1 

8 

S400103 

A 

5 

1 

1 

1 

9 

S400104 

C 

5 

1 

1 

1 

10 

S400105 

A 

5 

1 

1 

1 

11 

S400106 

C 

5 

1 

1 

1 

12 

S400108 

C 

5 

1 

1 

1 

13 

S400109 

A 

5 

1 

1 

1 

14 

S400110 

A 

5 

1 

1 

1 

15 

S400301 

C 

5 

1 

1 

1 

16 

S400302 

C 

5 

1 

1 

1 

17 

S400303 

C 

5 

1 

1 

1 

18 

S400304 

C 

5 

1 

1 

1 

19 

S400701 

A 

5 

4 

3 

2 

1 

M 

20 

S400702 

A 

A 

1 

2 

3 

4 

5 

M 

21 

S400703 

A 

1 

2 

3 

4 

5 

M 

22 

S400801 

F 

D 

5 

4 

3 

2 

1 

M 

23 

S400802 

F 

D 

5 

4 

3 

2 

1 

M 

24 

S400803 

F 

5 

4 

3 

2 

1 

M 

25 

S400804 

F 

5 

4 

3 

2 

1 

M 

26 

S400805 

F 

D 

5 

4 

3 

2 

1 

M 

27 

S400806 

F 

5 

4 

3 

2 

1 

M 

28 

S400901 

C 

5 

3 

.67 

2 

.33 

1 

M 

29 

S400902 

C 

5 

3 

.67 

2 

.33 

1 

M 

30 

S401201 

E 

B 

1 

3 

5 

M 

31 

S401202 

E 

B 

1 

3 

5 

M 

32 

S401203 

E 

1 

3 

5 

M 

33 

S401206 

E 

B 

1 

3 

5 

M 

34 

S401207 

E 

B 

1 

3 

5 

M 

35 

S401208 

E 

B 

1 

3 

5 

M 

36 

S401209 

E 

B 

1 

3 

5 

M 

37 

S401301 

A 

A 

5 

4 

3 

2 

1 

M 

38 

S401302 

A 

1 

2 

3 

4 

5 

M 

39 

S401303 

A 

A 

5 

4 

3 

2 

1 

M 

40 

S401304 

A 

A 

5 

4 

3 

2 

1 

M 
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Table  C.12 
(continued) 


COHORT 


1 

2 

3 

41 

S401305 

A 

A 

42 

S401401 

F 

43 

S401404 

F 

44 

S401501 

E 

45 

S401601 

B 

46 

S401602 

B 

G 

47 

S401603 

B 

48 

S401604 

B 

C 

49 

S401605 

B 

C 

50 

S401606 

C 

51 

S401607 

B 

c 

52 

S401608 

B 

c 

53 

S401801 

E 

54 

S401901 

A 

55 

S401902 

D 

3D 

S401905 

G 

3  / 

D 

->0 

D 

OA  n  o  n  m 

B 

E 

oU 

S40200Z 

B 

E 

C 1 

B 

E 

^  o 
b/. 

S4U^UU4 

B 

E 

o  J 

o /  no  nn  c 
b4U^UU3 

B 

E 

64 

S402101 

D 

A 

A 

65 

8402^01 

C 

G 

c  c 
66 

S402202 

D 

G 

6  / 

S402203 

C 

68 

S402204 

C 

by 

S402301 

B 

E 

/u 

S402302 

B 

E 

/I 

S402303 

B 

E 

72 

S402304 

B 

E 

73 

S402305 

E 

74 

S402306 

B 

D 

75 

S402307 

B 

E 

76 

S402401 

A 

A 

77 

S402501 

A 

A 

78 

S402502 

A 

A 

79 

S402503 

A 

A 

80 

S402701 

A 

A 

RESPONSES 


1 

2 

3 

4 

5 

5 

4 

3 

2 

1 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 

5 

4 

3 

2 

1 

5 

1 

5 

1 

5 

1 

5 

1 

5 

1 

5 

1 

5 

1 

5 

1 

1 

1 

,80 

2 

.60 

3.40 

4.20 

5 

1 

5 

1 

1 

5 

• 

5 

1 

5 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

•^ 

£. 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

1 

M 

5 

3 

.67 

2 

.33 

1 

5 

3 

.67 

2 

.33 

1 

5 

3 

.67 

2 

.33 

1 

5 

3 

.67 

2 

.33 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

1 

3 

1 

2 

3 

4 

5 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

5 

4 

3 

2 

1 

Table  C.12 
(continued) 


COHORT 


1 

2 

3 

81 

S402702 

A 

A 

82 

S402703 

A 

A 

83 

S402704 

A 

A 

84 

S402705 

A 

A 

85 

S402802 

D 

F 

86 

S402803 

D 

87 

S402804 

D 

F 

88 

S402807 

D 

F 

89 

S402808 

G 

90 

S402902 

B 

91 

S402904 

B 

92 

S402906 

B 

93 

S402908 

B 

94 

S403101 

G 

H 

95 

S403102 

G 

H 

96 

S403103 

G 

h 

97 

S403104 

G 

H 

98 

S403105 

H 

99 

S403106 

H 

100 

S403201 

B 

101 

S403301 

F 

102 

S403302 

F 

103 

S403306 

C 

104 

S403308 

C 

105 

S403401 

C 

106 

S403501 

G 

107 

S403502 

G 

108 

S403801 

E 

109 

S403901 

E 

110 

S404301 

D 

RESPONSES 


12345678  MISS 


5 

4 

3 

2 

1 

M 

5 

4 

3 

2 

1 

M 

1 

2 

3 

4 

5 

M 

5 

4 

3 

2 

1 

M 

5 

1 

M 

5 

1 

M 

5 

1 

M 

5 

1 

M 

1 

5 

M 

5 

3.67 

2.33 

1 

M 

5 

3.67 

2.33 

1 

M 

5 

3.67 

2.33 

1 

M 

5 

3.67 

2.33 

1 

M 

5 

3 

1 

M 

5 

3 

1 

M 

5 

3 

1 

M 

5 

3 

1 

M 

5 

3 

1 

M 

5 

3 

1 

M 

1 

2.33 

3.67 

5 

M 

M 

5 

3.67 

2.33 

1 

M 

5 

3.67 

2.33 

1 

M 

5 

3.67 

2.33 

1 

M 

5 

3.67 

2.33 

1 

M 

5 

1 

M 

M 

5 

1 

M 

5 

1 

M 

5 

3.67 

2.33 

1 

M 

M 

5 

3.67 

2.33 

1 

M 

M 

5 

1 

M 

M 
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APPENDIX  D 


The  IRT  Linking  Procedures  Used  to  Place 
the  1986  Intermediary  Scaling  Results 
onto  the  1984  Reading  Calibration  Scale 


Appendix  D 

THE  IRT  LINKING  PROCEDURE 
USED  TO  PLACE  THE  1986  INTERMEDIARY  SCALING  RESULTS 
ONTO  THE  1984  READING  CALIBRATION  SCALE 


Kathleen  M.  Sheehan 
Educational  Testing  Service 


The  item  calibration  program  employed  by  NAEP  resolves  the  indeterminacy 
of  scale  inherent  in  the  three -parameter  logistic  model  by  standardizing  the 
distribution  of  proficiency  in  the  calibration  sample.     As  a  result, 
intermediary  scaling  results,  such  as  item  parameters  and  conditional 
effects,  are  obtained  on  a  scale  that  is  approximately  centered  at  zero,  has 
a  standard  deviation  of  one,  and  typically  spans  a  region  lying  somewhere 
between  -4.0  and  4.0.     In  1984,  NAEP  elected  to  report  the  final  results  of 
the  reading  assessment  on  an  alternative  scale  that  was  selected  for  ease  of 
interpretation.     The  relationship  between  the  alternative  scale,  called  the 
RP  scale,  and  the  scale  established  by  the  calibration  of  the  1984  spiralled 
reading  data,  called  the  1984  reading  calibration  scale,  is  virtually  linear 
for  proficiencies  in  the  region  bounded  by    -4.0  and  4.0.     Thus,  scaling 
results  reported  on  the  RP  scale  can  be  reexpressed  on  the  1984  reading 
calibration  scale  by  applying  an  appropriate  linear  transformation. 

Because  the  RP  scale  was  primarily  developed  to  enhance  the 
interprecability  of  the  final  scaling  results  of  the  1984  reading  assessment, 
only  the  final  assessment  results  were  reported  on  that  scale.  The 
intermediary  results,  such  as  item  parameters  and  conditional  effects,  were 
reported  on  the  1984  reading  calibration  scale.     This  practice  of  reporting 
intermediary  scaling  results  on  a  calibration  scale  and  final  scaling  results 
on  a  specially  selected  reporting  scale  was  repeated  for  all  of  the  subject 
areas  assessed  in  1986.     For  example,  the  mathematics  and  science  item 
parameters  are  reported  on  the  1986  mathematics  calibration  scale  and  the 
1986  science  calibration  scale,  respectively. 

To  facilitate  year-to-year  comparisons,  the  1986  reading  calibration 
scale  was  equated  to  the  1984  reading  calibration  scale  and  all  of  the 
intermediary  results  of  the  1986  reading  assessment  were  reported  on  the  1984 
reading  calibration  scale.     The  IRT  linking  procedure  that  was  used  to  equate 
the  two  scales  and  the  jackknife  procedure  that  was  used  to  approximate  the 
equating  error  are  documented  in  this  appendix. 


TYPES  OF  IRT  LINKING  PROCEDURES 

Icem  parameter  estimates  obtained  from  independent  item  calibrations  can 
be  placed  onto  a  common  scale  by  estimating  an  appropriate  linking 
transformation.     Two  commonly  used  procedures  for  estimating  IRT  linking 
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transformation    are  common  item  equating  and  equivalent  population  equating. 
Coimnon  item  equating  procedures  are  used  when  the  scales  to  be  equated 
contain  overlapping  subsets  of  items  that  were  administered  to  independent 
samples  of  examinees  drawn  from  (possibly)  different  populations.  Equivalent 
population  equating  procedures  are  used  when  the  scales  to  be  equated  contain 
nonover lapping  subsets  of  items  that  were  administered  to  independent  samples 
of  examinees  drawn  from  equivalent  populations. 

Since  introducing  scale-score  reporting  in  1984,  NAEP  has  had  occasion 
to  use  both  of  the  linking  procedures  described  above.     In  1984,  NAEP  used  an 
equivalent  population  equating  procedure  to  link  the  scale  established  by  the 
calibration  of  the  paced  reading  data  and  the  scale  established  by  the 
calibration  of  the  spiralled  reading  data.     The  equivalent  population 
procedure  could  be  used  because,  at  each  age  level,  the  sample  of  students 
who  received  paced  booklets  was  constructed  to  be  randomly  equivalent  to  the 
sample  of  students  who  received  spiralled  booklets.     Note  that  although  the 
paced  and  spiralled  booklets  contained  many  of  the  same  items,  those  items 
did  not  qualify  as  common  items  because  they  were  administered  under 
different  conditions  and  the  difference  in  conditions  was  expected  to  be 
related  to  reading  proficiency.     The  equivalent  population  procedure  used  to 
link  the  paced  and  spiralled  scales  is  documented  in  Beaton  (T^S7a). 

NAEP's  first  use  of  common  item  equating  occurred  in  1986  at  the  time 

that  the  1986  reading  calibration  scale  was  equated  to  the  1984  reading 

calibration  scale.     The  particular  procedures  used  for  that  equating  are 
documented  in  this  appendix. 

Note  that  the  reading  trend  analysis  documented  in  Beaton  (1987a)  did 
not  involve  any  scale  equating  because  the  1984  reading  data  and  the  reading 
trend  data  from  all  previous  years  were  calibrated  simultaneously. 

THE  IRT  LINKING  PROCEDURE  USED  TO  EQUATE  THE  1984  AND  1986  CALIBRATION  SCALES 

As  in  the  1984  assessment,  the  1986  reading  results  were  calibrated  with 
the  BILOG  program  (Mislevy  &  Bock,  1982).     The  calibration  included  a  total 
of  107  cognitive  reading  items,  76  of  which  were  common  to  the  1984 
assessment  and  31  of  which  were  administered  for  the  first  time  in  1986. 
Although  appropriately  scaled  item  parameters  were  already  available  for  the 
76  common  items  and  those  parameters  were  used  in  all  subsequent 
calculations,  these  items  were  recalibrated  with  the  1986  data  so  that  a 
second  set  of  item  parameters  might  be  obtained.     The  two  sets  of  item 
parameters  available  for  the  common  items  were  then  used  to  define  an 
appropriate  linking  transformation. 

As  is  typical  in  a  BILOG  calibration,  the  1986  item  parameters  were 
obtained  on  a  provisional  scale  that  was  determined  by  standardizing  the 
distribution  of  proficiency  in  the  calibration  sample.     (The  calibration 
sample  included  students  from  three  age  groups :  grade  3/age  9 ,   grade  7/age 
13,  and  grade  11/age  17.) 
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The  Stocking-Lord  linking  procedure  (Stocking  &  Lord,  1983),  as 
implemented  in  the  TBLT  program  (Stocking,  1986),  was  used  to  estimate  the 
linking  transformation  needed  to  reexpress  the  1986  intermediary  scaling 
results  on  the  1984  calibration  scale. 

The  input  data  for  the  Stocking-Lord  procedure  consists  of  two  sets  of 
estimated  item  parameters,  one  set  expressed  on  a  target  scale,  and  one  set 
expressed  on  a  provisional  scale.     In  the  application  documented  here,  the 
1984  reading  calibration  scale  is  the  target  scale  and  the  1986  reading 
calibration  scale  is  the  provisional  scale.     The  output  of  the  Stocking-Lord 
procedure  is  the  parameter  estimates,  A  and  B,  of  the  linear  transformation 
that  describes  the  relationship  between  the  IRT  item  parameter  estimates 
expressed  on  the  provisional  scale  and  those  expressed  on  the  target  scale. 
That  is 

a.  -  A-^a.P 
J  J 

b.  «  A  b.^  +  B 
J  J 

p 

c .  «  c . 
J  J 

where  (a^,b^,c^)  and  (a    b    c  )  for  j=l,...n  are  the  IRT  item  parameter 
estimates  obtained  for  the*^ common  items  expressed  on  the  provisional  and 
target  scales,   respectively.     Note  that  the  lower  asyroptote  parameters,  c.^, 
are  unaffected  by  the  transformation.  J 

The  parameters  of  the  linear  transformation,  A  and  B,  are  found  by 
minimizing  the  squared  difference  between  estimated  true  scores  (expected 
numbers  correct  on  the  n  common  items)  at  N  preselected  proficiency  values 
^«[^^,...^  ].     The  function  that  is  minimized  is 


f(A,B)  '    1/N    S     {f^(l,0,^?.)   -  cP(A,B,^?.)}^ 
i«l  ^  ^ 

T 

where  f  (1,0,^^)  is  the  estimated  true  score  associated  with  the  proficiency 
level^   ,  calculated  from  the  item  parameters  expressed  on  the  target  scale, 
and  ^  (A,B,^^)  is  the  estimated  true  score  associated  with  the  proficiency 
level  6^  calculated  from  the  item  parameters  that  were  originally  estimated 
on  the  provisional  scale  and  then  reexpressed  on  the  target  scale,  that  is 


where  a.     and  b.     are  the  estimated  discrimination  and  difficulty  parameters 
for  item  j,  expressed  on  the  provisional  saale.     The  values  d^[d^  ,6^, , . .6  ] 
are  typically  selected  to  span  that  region  of  the  target  scale  wnich  is  ^ 
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expected  to  be  most  dense.     For  the  current  application,  200  equally  spaced 
values  between  -2.0  and  2.0  were  used. 

The  success  of  the  equating  was  evaluated  by  comparing  estimated  item 
difficulties  and  test  characteristic  curves  calculated  from  the  common  items. 
The  comparison  of  item  difficulties  (b  parameters)  is  provided  in  Figure  D.l. 
The  transformed  1986  values  are  plotted  on  the  vertical  axis,  the 
corresponding  1984  values  are  plotted  on  the  horizontal  axis.    The  plot  shows 
that  the  parameter  estimates  obtained  from  the  TBLT  run  have  provided  an 
adequate  fit  to  the  data. 

The  test  characteristic  curve  provides  the  expected  proportion  of  items 
passed  as  a  function  of  examinee  proficiency.     Figure  D,2  presents  test 
characteristic  curves  calculated  for  the  test  defined  by  the  76  common  items 
only.     The  curve  defined  by  the  item  parameters  that  were  estimated  from  the 
1984  data  is  plotted  as  a  solid  line.     The  curve  defined  by  the  item 
parameters  that  were  estimated  from  the  1986  data  is  plotted  as  a  dashed 
line.     The  comparison  indicates  that,  at  almost  all  proficiency  levels,  the 
linking  transformation  has  been  successful  at  preserving  the  expected 
proportion  of  items  passed.     The  difference  in  the  curves  at  the  lower 
asymptote  reflects  the  fact  that  the  guessing  parameters,  c,  are  generally 
not  well  estimated.  ^ 

Based  on  the  results  given  above,  it  was  decided  that  all  intermediary 
scaling  results,  such  as  item  parameters  and  conditional  effects,  could  be 
placed  onto  the  1984  calibration  scale  by  applying  the  linking  transformation 
that  was  obtained  from  the  TBLT  run.     Note  however  that  only  the  31  items 
that  were  first  administered  in  1986  needed  to  be  transformed.     For  the  76 
common  items,  item  parameters  expressed  on  the  1984  calibration  scale  were 
already  available  and  so  no  rescaling  was  required. 

THE  UNCERTAINTY  OF  THE  LINKING  PROCEDURES 

The  uncertainty  associated  with  the  parameter  estimates  A  and  B  of  the 
linking  transformation  was  approximated  using  a  jackknife  procedure 
(Hosteller  &  Tukey,  1977).     The  procedure  consisted  of  three  steps.  First, 
the  76  items  that  were  used  to  estimate  the  transformation  were  classified 
into  10  subsets  with  approximately  equal  average  difficulty.     (Six  subsets 
contained  eight  items  and  four  subsets  contained  seven  items.)     Second,  the 
TBLT  program  was  rerun  10  times.     Each  rerun  included  all  but  one  of  the 
subsets  created  in  step  1.     Finally,  the  observed  variation  among  the  A  and  B 
parameter  estimates  obtained  from  the  10  TBLT  reruns  was  used  to  estimate  a 
covariance  matrix  which  quantifies  uncertainty  due  to  (i)  the  imprecision  of 
the  estimated  item  parameters  ar.d  (ii)  lack  of  fit  from  the  IRT  model.  The 
results  of  this  analysis  are  given  in  Table  D.l. 

In  considering  the  magnitude  of  the  variance  estimates  given  in  Table 
D.l,  recall  that  the  jackknife  procedure  measures  variation  arising  from  two 
different  sources:     estimation  error  and  model  misfit.     The  magnitude  of  the 
uncertainty  due  to  estimation  error  is  related  to  the  size  of  the  calibration 
samples --larger  calibration  samples  tend  to  provide  more  precise  linking 
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Figure  D.l 

Comparison  of  Item  Difficulties  for  76  Common  Items 
1986  b's  vs.  1984  b's 

Scale  -  1984  Calibration  Scale 
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Figure  D.2 

Comparison  of  Test  Characteristic  Curves  for  76  Common  Items 

1984  Curve  -  Solid  Line 
1986  Curve  -  Dashed  Line 


c 


3 


PROFICIENCY 


ERLC 


-560- 

603 


Table  D.l 


Results  of  the  Jackknife  Approximation 
for  the  Stocking-Lord  Linking  Procedure 


Rim 

I  teins 

A 

B 

n 

\j 

/  o 

1  . 

-u 

442910 

1 

68 

1. 

118018 

-0 

449670 

2 

68 

1 

126296 

-0 

447837 

3 

68 

1 

121856 

-0 

449472 

4 

68 

1 

110982 

-0 

433893 

5 

68 

1 

114703 

-0 

426793 

6 

68 

1 

128065 

-0 

430320 

7 

69 

1 

125834 

-0 

446748 

8 

69 

1 

128753 

-0 

440663 

9 

69 

1 

112862 

-0 

447648 

10 

69 

1. 

135424 

-0 

455858 

Jackknife 
Parameter  Estimate 

0.00512 

0/00740 

0.00238 


^  A 

2 
^  B 

%3 


The  parameter  estimates,  A  and  B,  obtained  from  Run  0  were  used  to 
transform  the  item  parameters  estimated  from  the  1986  data  from  the  metric 
of  the  1986  calibration  scale  to  the  metric  of  the  1984  calibration  scale. 
The  parameter  estimates,  A  and  B,  obtained  from  Runs  1  through  10  were  used 
only  to  estimate  the  uncertainty  of  the  linking  procedure. 
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parameter  estimates.     The  magnitude  of  the  uncertainty  due  to  model  misfit  is 
related  to  the  number  of  linking  items  used  to  define  the  transformation.  To 
see  this,  note  that  if  the  IRT  model  were  correct,  the  difference  between 
sets  of  (a,b,c)  estimates  obtained  from  increasingly  large  samples  of 
examinees  would  be  accounted  for  totally  by  a  linear  transformation.     In  this 
case,  precise  estimates  of  the  linking  parameters  could  be  obtained  with  as 
few  as  two  linking  items.     When  the  IRT  model  does  not  fit,  however, 
different  sets  of  linking  items  will  tend  to  provide  different  estimates  of 
the  linking  parameters  even  as  the  calibration  sample  sizes  increase  without 
bound.     In  this  latter  case,  it  is  clear  that  the  model  misfit  component  of 
uncertainty  can  only  be  reduced  by  increasing  the  number  of  linking  items. 
Moreover,  the  linking  items  should  be  chosen  so  as  to  be  representative  of 
the  set  of  all  items  that  might  have  been  used  to  estimate  the  linking 
function. 

Because  the  linking  transformation  described  in  this  appendix  was  used 
only  to  transform  intermediary  scaling  results,  the  variance  estimates  given 
in  Table  D.l  are  not  particularly  interesting.     However,  if  the  final  results 
of  the  reading  assessment  had  been  reported  on  the  same  scale  as  was  used  in 
1984,  as  may  be  the  case  in  future  reading  assessments,   then  these  types  of 
variance  estimates  could  be  used  to  account  for  equating  error  when 
estimating  standard  errors  for  statistics  measu.ring  year-to-year  changes  in 
individual  or  group  performance. 

Some  consequences  of  the  uncertainty  associated  with  the  Stocking-Lord 
linking  procedure  are  reported  in  Sheehan  and  Mislevy  (1988).     In  particular 
it  is  shown  that  the  effects  of  linking  procedure  uncertainty  will  usually  be 
negligible  for  inferences  drawn  about  individuals  but  can  be  quite 
substantial  for  inferences  based  on  aggregate  statistics  such  as  group  means. 

Because  the  plausible  values  methodology  used  to  scale  the  NAEP  data 
does  not  allow  for  inferences  at  the  individual  level,  the  issue  of  the 
consequences  of  uncertainty  for  individual  inferences  will  not  be  addressed 
in  this  appendix. 

The  following  section  provides  a  detailed  description  of  a  procedure 
that  can  be  used  to  account  for  linking  procedure  uncertainty  when  drawing 
inferences  about  aggregate  statistics  such  as  changes  in  group  means. 

THE  CONSEQUENCES  OF  UNCERTAINTY  FOR  INFERENCES  ABOUT  GROUP  MEANS 

Consider  the  problem  of  estimating  the  change  in  mean  reading 
proficiency  for  a  particular  population  subgroup  over  the  two  year  period 
from  1984  to  1986.      Let  /Ig^  represent  the  estimated  mean  reading  proficiency 
of  that  subgroup  calculated  from  the  1984  scaling  results  expressed  on  the 
1984  calibration  scale.     Let  {i^^  represent  the  estimated  mean  reading 
proficiency  for  that  same  subgroup  calculated  from  the  1986  scaling  results 
expressed  on  the  1986  calibration  scale  and  let  Mg^^.  represent  the  1986  mean 
reexpressed  on  the  1984  calibration  scale.     Since  /i^,  and  f^or    are  both 


expressed  on  the  same  scale,  an  estimate  of  the  change  from  to  1986  can 

be  obtained  from  the  difference,  D«Mq,  'Aq/ •     If  it  is  assumed  that  the 
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parameters  of  the  linking  transformation  are  known  without  error,  then  the 
standard  error  of  this  estimate  is  given  by 


SE(D)  .  SEC,,,^  -  ^84)  -  (o\^^  ^  a\^)^/^ 


(1) 


where  a^^  is  the  standard^ error  of  expressed  on  the  1984  scale,  and  o 
is  the  standard  error  of  Ag^^  expressed  on  the  equated  1986  scale. 


To  account  for  the  uncertainty  of  the  linking  procedure  used  to 
reexpress  the  1986  results  on  the  1984  scale,  note  that  /i ^    -  A  p,^^  +  B  and 
the  standard  error  of  /l^^,  denoted  o^^,  is  not  affected  by  ihe  uncertainty  of 
the  linking  procedure. 


A  large  sample  approximation  for  the  desired  standard  error  can  be 

obtained  by  first  defining  a  covariance  matrix  for  [Ao^*  A»  S]  as  follows 

oo 


86 


0 
0 


0 

2 

'  A 


AB 
2 
'  B 


where  a^,  a^,  and         quantify  estimation  variation  for  the  parameters  A  and 
B  of  the  linking  transformation.     (The  quantities  a^,  a^,  and  <7      can  be 
approximated  using  the  jackknife  procedure  that  was  given  in  the  previous 
section.)     Second,  note  that 


-  Var(g(Agg.  A,  6)) 


86 


dB 


dA 


as 


"  f  ^'  '^86'  1  1  2  [  A.  Mgg.  1  ] 
'  A^a^g^  +  <'^86>^'^^6  *  ^f'SG^AB  ^ 
-  f(/igg.A.S) 


«  f(Agg.A,S) 


(2) 
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Thus,  the  uncertainty  associated  with  the  linking  procedure  can  be  accounted 
for  in  the  estimated  standard  error  of  the  difference  as  follows 


SE(D)  -  SE(A86^  -  ^84^ 

-  (  Var(M86,)  ^  o\^)^/^ 

-  (f(A8g,A,S)  +  ^^84)^/^  (3) 

where  f(/lo^,A,S)  is  given  as  in  (2). 
00 

A  NUMERICAL  ILLUSTRATION 

Mean  reading  proficiencies  for  students  aged  9,  13,  and  17  in  1984  and 
1986  are  given  in  Table  D.2.     The  first  row  of  the  table  provides  1984  age 
group  means  expressed  on  the  1984  calibration  scale.     The  second  and  third 
rows  provide  1986  age  group  means  expressed  on  the  1986  calibration  scale  and 
the  1984  calibration  scale.     The  table  also  provides  estimated  standard 
errors  for  each  mean. 

Table  D.2 
Mean  Reading  Proficiencies 
With  Standard  Errors  in  Parentheses 


Year  Scale                     Age  9  Age  13  Age  17 

1984  84  Calib.  -0.752(.020)  0.150(.014)  0.766(.018) 

1986  86  Calib.  -0.375(.025)  0.571(.019)  0.874(.018) 

1986  84  Calib.  -0.864(.028)  0.198(.022)  0.538(.020) 


The  change  in  mean  reading  proficiency  from  1984  to  1986  is  given  in 
Table  D.3.     The  two  different  methods  available  for  calculating  the  standard 
errors  of  these  statistics  are  compared  in  Table  D.4.     To  put  these  results 
in  perspective,  Table  D.5  provides  the  change  in  mean  reading  proficiency 
expressed  in  standard  error  units.     The  table  shows,  for  example,  that  the 
decrease  in  the  mean  reading  proficiency  of  9-year-olds  is  approximately 
three  standard  errors  when  the  uncertainty  of  the  linking  procedure  is  not 
accounted  for,  but  only  one  standard  error  when  it  is. 


Table  D.3 

The  Change  in  Mean  Reading  Proficiency  from  1984  to  1986 

Scale  Age  9  Age  13  Age  17 

84  Calib.  -0.112  0.048  -0.228 
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Table  D.4 


Approximate  Standard  Errors 
for  the  Change  in  Mean  Reading  Proficieticy  from  1984  to  1986 

★ 

Method  Age  9  Age  13  Age  17 

Excl.  Eq.  Error  .034  .026  .027 

Incl.  Eq.  Error  .105  .084  .066 


Table  D.5 

The  Change  in  Mean  Reading  Proficiency  from  1984  to  1986 
Expressed  in  Standard  Error  Units 


Standard  Error  Age  9  Age  13  Age  17 

Excl.  Eq.  Error  -3.29  1.85  -8.44 

Incl.  Eq.  Error  -1.07  .57  -3.45 


Excl.  Eq.  Error  refers  to  the  method  that  assumes  that  the  linking  function 
is  known  without  error,  as  in  equation  (1);     Incl.  Eq .  Error  refers  to  the 
method  that  accounts  for  the  uncertainty  of  the  linking  procedure  as  in 
equation  (3). 


-565- 


APPENDIX  E 
IRT  Item  Parameter  Tables 


613 


Appendix  £ 
IRT  ITEM  PARAMETER  TABLES 


Appendix  E  contains  36  tables  of  IRT  parameters  for  NAEP  items  used  in 
IRT  scaling  for  each  grade/age  or  age. 

For  each  NAEP  item  (FIELD) ,  the  tables  show  the  BLOCK  in  which  the  item 
appears  (for  the  grade/age  or  age),  the  order  in  which  the  item  appears 
within  the  block  (ITEM),  the  corresponding  IRT  parameters  (A,  B,  and  C) ,  the 
standard  error  (SE)  for  each  parameter,  and  a  short  ITEM  DESCRIPTION. 

Tables  E.l  through  E.28  show,  for  each  grade/age,  IRT  parameters  for 
reading  items,  mathematics  items  (by  subscale) ,  and  science  items  (by 
si?hscale).     Tables  E.29  and  E.30  present  IRT  parameters  for  grade  11/age  17 
fo-   U.S.  history  items  and  literature  items,  respectively. 

Tables  E.31  through  E.36  show  IRT  parameters  for  ages  9,  13,  and  17  for 
Items  used  in  mathematics  and  science  trend  scaling. 

Not.    that  item  parameters  shown  in  this  appendix  are  in  the  metrics  used 
for  clie  c  riginal  calibration  of  the  scale  or  subscale.     The  transformations 
needed  tv  represent  these  parameters  in  terms  of  the  metric  of  the  final 
reporting  scales  are  given  in  Chapters  9,  10,  11,  and  13,  respectively,  for 
.-wading,  mathematics,  science,  and  history  and  literature. 
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Table  E.l 

1986  IRT  Parameters,  Reading,  Grade  3/Age  9 


FIELD 

BLOCK 

ITEM 

A 

SE 

♦NOlOAOl 

Rl 

12 

0 

715 

(0 

.087 

*N010A02 

Rl 

13 

0 

928 

(0 

.  171 

*N010A03 

Rl 

14 

1 

031 

(0 

.  197 

*N008901 

Rl 

15 

1 

328 

(0 

.  106 

*N008902 

Rl 

16 

1 

258 

(0 

.  102 

*N001501 

Rl 

17 

1 

808 

(0 

.  130 

*N001502 

Rl 

18 

1 

643 

(0 

.098 

*N001503 

Rl 

19 

1 

345 

(0 

.088 

*N00150A 

Rl 

20 

1 

448 

(0 

.089 

*N01A201 

Rl 

21 

1 

207 

(0 

.  134 

*N010301 

R2 

10 

0 

702 

CO 

.085 

*N009801 

R2 

11 

1 

396 

(0 

.  134 

*N013301 

R2 

12 

1 

232 

(0 

.  161 

*N009401 

R2 

13 

1 

882 

(0 

.  127 

*N002001 

R2 

14 

1 

197 

(0 

.065 

*N002002 

R2 

15 

1 

444 

(0 

.084 

*N002003 

R2 

16 

1 

583 

(0 

.093 

♦N002801 

R2 

17 

1 

921 

(0 

.114 

♦N002802 

R2 

18 

1 

896 

(C 

.  110 

*N00A101 

R2 

19 

1 

096 

(0 

.087 

*N010201 

R2 

20 

1 

243 

(0 

.  121 

*N010501 

R3 

8 

2 

023 

(0 

.  139 

*N010502 

R3 

9 

1 

204 

(0 

.  114 

*N010503 

R3 

10 

1 

455 

(0 

.  123 

*N01050A 

R3 

11 

2 

300 

(0 

.  166 

♦N003101 

R3 

12 

1 

571 

(0 

.  100 

*N003102 

R3 

13 

1 

530 

(0 

.083 

*N00310A 

R3 

14 

0 

704 

(0 

.  042 

*N008601 

R3 

15 

1 

789 

(0 

.  179 

*N008602 

R3 

16 

1 

368 

(0 

.  179 

*N008603 

R3 

17 

1 

206 

(0 

.  118 

N021101 

R4 

5 

0 

800 

(0 

.  024 

N021102 

R4 

6 

0 

760 

(0 

.  030 

N0211C3 

R4 

7 

1 

271 

(0 

.  038 

N021301 

R4 

8 

1 

081 

(0 

.  024 

N021303 

R4 

10 

0 

994 

(0 

.  031 

N021304 

R4 

11 

0 

446 

(0 

.017 

N021305 

R4 

12 

0 

978 

(0 

.  024 

N021201 

R4 

13 

0 

861 

(0 

.  024 

N021202 

R4 

14 

0 

628 

(0 

.  021 

N021203 

R4 

15 

0 

704 

(0 

.  022 

N021204 

R4 

16 

0 

726 

(0 

.  022 

*N010102 

R5 

12 

1 

124 

(0 

.  193 

♦N010103 

R5 

13 

1 

795 

(0 

.  200 

N021<i01 

R5 

14 

0 

958 

(0 

.043 

N021402 

R5 

15 

1 

764 

(0 

.  077 

N021403 

R5 

16 

1 

650 

(0 

.  041 

N021404 

R5 

17 

2 

362 

(0 

.225 

*N014301 

R5 

18 

1 

755 

(0 

.  191 

*N01A302 

R5 

19 

1 

074 

(0 

.  136 

♦N01A303 

R5 

20 

1 

721 

(0 

.  187 

*N013001 

R5 

21 

1 

020 

(0 

.  122 

*N013002 

R5 

22 

0 

972 

(0 

.  121 

*N013003 

R5 

23 

1 

717 

(0 

.  164 

*N01300A 

R5 

24 

0 

994 

( 0 

.  115 

N021501 

R6 

5 

0 

991 

(0 

.036 

N021502 

R6 

6 

1 

524 

(0 

.038 

N021503 

R6 

7 

1 

100 

(0 

.028 

N02150A 

R6 

8 

1 

040 

(0 

.039 

N021505 

R6 

9 

1 

572 

(0 

.057 

♦K010601 

R6 

10 

1 

604 

(0 

.  196 

*N010602 

R6 

11 

1 

788 

(0 

.344 

♦N010603 

R6 

12 

1 

359 

(0 

.  199 

*N010604 

R6 

13 

1 

637 

(0 

.250 

♦N010605 

R6 

14 

1 

220 

(0 

.  190 

*N013401 

R6 

15 

1 

203 

(0 

.  177 

*N013402 

R6 

16 

1 

438 

(0 

.  189 

*N013A03 

R6 

17 

1 

494 

(0 

.223 

♦Used  in  1984  reading  scale. 


B  SE  C  SE 


-1 

.487 

( 0 

209 ) 

0 

219 

( 0 

07  7) 

0 

132 

( 0 

113 ) 

Q 

222 

0 

465 

(0 

153 ) 

0 

190 

f  n 

n  9  7  ^ 

-1 

244 

(0 

138) 

0 

148 

( 0 

041) 

-1 

271 

( 0 

140 ) 

0 

156 

f  n 

U  *f  J  y 

~1 

313 

( 0 

152) 

Q 

225 

f  n 

n  A  7  ^ 

-0 

507 

( 0 

061 ) 

n 
u 

1  ft9 

f  n 

n  9  R  \ 

-0 

902 

(0 

086) 

Q 

207 

f  n 

n  A  ^ 

U  *f  J  y 

-0 

650 

( 0 

068 ) 

Q 

17  3 

f  n 

n  Q  9  \ 

U  J  Z  y 

-1 

218 

( 0 

189 ) 

n 
u 

X  J  D 

e  n 

n  «\9  \ 

U  DZ  y 

~2 

383 

( 0 

318) 

u 

9  A  ft 

I  n 
K  w 

093) 

-2 

227 

( 0 

296 ) 

Q 

2  59 

n  ft  R  ^ 

U  O  D  y 

"1 

.  557 

( 0 

268 ) 

Q 

253 

(  0 

07  7) 

- 1 

402 

( 0 

172 ) 

u 

r  ti 

n  <5  \ 

~0 

013 

( 0 

050 ) 

u 

1 J 1 

r  n 

n  9  n  \ 
uzu  ) 

-0 

042 

( 0 

055 ) 

203 

(  0 

020  ) 

-0 

229 

( 0 

054  ) 

Q 

224 

n  9  9  \ 
uzz  ) 

~0 

767 

( 0 

081 ) 

u 

1  /  D 

(  0 

028  ) 

~0 

912 

( 0 

092 ) 

n 
u 

ISO 

n  9  ft  ^ 
u  z  O  y 

-1 

122 

(0 

114  ) 

0 

229 

C  0 

054  ) 

-1 

932 

( 0 

245 ) 

0 

24  4 

C  0 

078) 

-1 

490 

(0 

190 ) 

0 

204 

C  0 

04  6) 

-1 

196 

( 0 

154  ) 

0 

156 

( 0 

04  9) 

-1 

460 

(0 

184  ) 

0 

159 

( 0 

04  8) 

"1 

114 

(0 

174  ) 

0 

182 

( 0 

032 ) 

-0 

645 

(0 

073  ) 

0 

267 

C  0 

0  32) 

-0 

359 

( 0 

051 ) 

0 

14  5 

C  0 

023 ) 

1 

923 

(0 

.124  ) 

0 

0 

( 0 

0  ) 

-0 

972 

(0 

171 ) 

0 

169 

( 0 

037 ) 

-0 

554 

(0 

122) 

0 

261 

( 0 

041) 

~0 

985 

(0 

137  ) 

0 

14  0 

C  0 

043 ) 

-2 

200 

(0 

064) 

0 

214 

(0 

021 ) 

"1 

407 

(0 

050) 

0 

188 

(0 

015) 

"1 

461 

(0 

050) 

0 

181 

( 0 

015 ) 

-0 

168 

(0 

019) 

0 

0 

( 0 

0  ) 

-0 

745 

(0 

024  ) 

0 

190 

( 0 

010 ) 

0 

397 

(0 

037  ) 

0 

172 

( 0 

008 ) 

0 

214 

(0 

028) 

0 

189 

( 0 

007  ) 

-0 

120 

(0 

023) 

0 

180 

( 0 

008 ) 

-0 

153 

(0 

022) 

0 

195 

(0 

009) 

0 

113 

(0 

027  ) 

0 

204 

(0 

008 ) 

-0 

262 

(0 

021 ) 

0 

190 

( 0 

009 ) 

"0 

050 

(0 

111) 

0 

267 

C  0 

03  7) 

-1 

075 

(0 

207  ) 

0 

209 

(0 

042) 

-0 

798 

(0 

039) 

0 

137 

C  0 

Oil ) 

-0 

878 

(0 

050) 

0 

141 

(0 

010 ) 

-1 

625 

(0 

059) 

0 

221 

(0 

016) 

-0 

410 

(0 

065) 

0 

181 

(0 

008 ) 

-0 

820 

(0 

158) 

0 

190 

(0 

035) 

-0 

498 

(0 

108) 

0 

181 

(0 

041) 

-1 

041 

(0 

188) 

0 

208 

(0 

041) 

-0 

343 

(0 

083) 

0 

165 

(0 

036) 

-0 

383 

(0 

090) 

0 

187 

(0 

040 ) 

-1 

123 

(0 

172) 

0 

234 

CO 

041) 

-0 

946 

(0 

138) 

0 

216 

CO 

056) 

-1 

340 

(0 

049) 

0 

196 

CO 

014) 

-1 

785 

(0 

060) 

0 

229 

CO 

018) 

-2 

234 

(0 

063) 

0 

214 

CO 

023) 

-1 

280 

(0 

050) 

0 

197 

CO 

014) 

-1 

117 

(0 

052) 

0 

166 

CO 

012) 

-0 

634 

(0 

136) 

0 

246 

CO 

036) 

0 

209 

(0 

153) 

0 

306 

CO 

023) 

-0 

258 

(0 

105) 

0 

234 

CO 

033) 

-0 

101 

(0 

106) 

0 

235 

CO 

026) 

-0 

069 

(0 

098) 

0 

184 

CO 

031) 

-0 

250 

(0 

107) 

0 

157 

CO 

035) 

-0 

862 

(0 

175) 

0 

205 

CO 

048) 

-0 

278 

CO 

116) 

0 

199 

CO 

033) 
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ERIC 


617 


Table  E.2 

1986  IRT  Parameters,  Reading,  Grade  7/Age  13 


FIELD  BLOCK 

TTFM 

SE 

Q 

SE 

c 

SE 

*N002801 

Rl 

1 

1 . 

Q9  1 

(0. 

llA) 

-Q 

767 

(0. 

081) 

0 . 

175 

(0. 

028) 

*N002802 

Rl 

9  1 

1 

896 

(0. 

110) 

"0  . 

9 12 

(0. 

092) 

0 . 

1  A3 

(0. 

028) 

*N002001 

Rl 

9  9 

1  Q7 

(0. 

065) 

"0 . 

013 

(0. 

050) 

0  . 

131 

(0. 

020) 

*N002002 

Rl 

23 

1  ^ 

AAA 

(0. 

08A) 

-0 . 

0  A2 

(0. 

055) 

0 . 

203 

(0. 

020) 

*K002003 

Rl 

2A 

1  ^ 

583 

(0. 

093) 

-0 . 

229 

(0. 

05A) 

0 . 

22A 

(0. 

022) 

*N001501 

Rl 

25 

1  ^ 

808 

(0. 

130) 

- 1  ^ 

313 

(0. 

152) 

0  . 

225 

(0. 

0A7) 

*N001502 

Rl 

26 

1  ^ 

6A3 

(0. 

098) 

-0 . 

507 

(0. 

061) 

0 . 

182 

(0. 

026) 

*N001503 

Rl 

97 

1 . 

3A  5 

(0. 

088) 

"0 . 

902 

(0. 

086) 

0 . 

207 

(0. 

OAS) 

*N00150A 

Rl 

79 

1  ^ 

A  A8 

(0. 

089) 

-0 . 

650 

(0. 

068) 

0  . 

173 

(0. 

032) 

*N003101 

Rl 

29 

1  ^ 

57 1 

(0. 

100) 

-0 . 

6A  5 

(0. 

073) 

0  . 

267 

(0. 

032) 

*N003102 

Rl 

30 

1  ^ 

530 

(0. 

083) 

-0 . 

359 

(0. 

051) 

0  . 

1 A5 

(0. 

023) 

*N00310A 

Rl 

3 1 

0  ^ 

70  A 

(0  . 

0A2) 

1  ^ 

923 

(0. 

12A) 

0  . 

0 

(0. 

0  ) 

*N008201 

R2 

10 

2 . 

72A 

(0. 

302) 

-0 . 

A71 

(0. 

131) 

0  . 

323 

(0. 

0  52) 

*N008202 

R2 

1  ^ 

1A6 

(0. 

102) 

-0 . 

065 

(0. 

102) 

0  . 

188 

(0. 

052) 

*N008203 

R2 

12 

1  ^ 

5A  3 

(0. 

lAl) 

-0  . 

289 

(0. 

lOA) 

0  . 

2A7 

(0. 

05A) 

*N00820A 

R2 

13 

0 

Du  u 

(0. 

236) 

~  0 . 

228 

(0. 

092) 

0 . 

209 

(0. 

038) 

*N008205 

R2 

1 A 

2 . 

1 A  5 

(0. 

188) 

-0  . 

256 

(0. 

092) 

0  . 

205 

(0. 

0A2) 

*N005001 

R2 

1  c 

1 D 

1 

99«3 

(0  . 

102) 

1  ^ 

380 

(0. 

159) 

0  . 

211 

(0 

Oil) 

*H005002 

R2 

ID 

u  . 

OD9 

(0. 

108) 

9  ftft 

(0. 

2A0) 

9  R  A 

(0. 

029) 

*N005003 

R2 

1  / 

u  . 

7  Q  7 

(0. 

105) 

1 

on  *\ 

9U  D 

(0. 

331) 

Q 

135 

(0. 

02A) 

*K003001 

R2 

1  n 
1 0 

1 

9Q1 

Z90 

(0. 

109) 

1 

153 

(0. 

169) 

0  . 

207 

(0. 

013) 

*N003003 

R2 

1  Q 

9 

29A 

(0. 

109) 

1  ^ 

72A 

(0. 

190) 

0  ^ 

120 

(0. 

006) 

*N00A601 

R3 

1 

u  . 

RQ  Q 
099 

(0. 

078) 

n 

1  7  Q 
1/9 

(0. 

lOA) 

n 

18  A 

(0. 

0A8) 

*N00A602 

R3 

1  7 

1 

lift 

(0. 

103) 

-  Q  ^ 

085 

(0. 

092) 

0  . 

2A9 

(0. 

OAA  ) 

*N00A603 

R3 

1  ft 

1  ^ 

A  8  5 

(0 

113) 

-  0 . 

516 

(0. 

089) 

0 

226 

(0. 

05A) 

*N007A01 

RA 

0 
0 

1 

098 

(0 

076) 

0  ^ 

531 

(0. 

096) 

0 

123 

(0. 

027) 

*N007A02 

RA 

Q 

1 

30  A 

(0 

08A) 

u 

117 

(0. 

075) 

Q 

176 

(0 

0A5) 

*N007A03 

RA 

1  n 

1 

756 

(0 

119) 

Q 

2 1 A 

(0 

093) 

0 

233 

(0 

027) 

*N007iiOA 

RA 

1  1 

Q 

985 

(0 

072) 

0 

060 

(0 

088) 

0 

181 

(0 

OAA  ) 

*N007A05 

RA 

12 

Q 

887 

(0 

102) 

1 

AOl 

(0 

229) 

0 

187 

(0 

025) 

*N007301 

RA 

xo 

1 

183 

(0 

091) 

-0 

39A 

(0 

100) 

0 

278 

(0 

059) 

*N007302 

RA 

lA 

0 

818 

(0 

059) 

0 

285 

(0 

08A) 

0 

136 

(0 

039) 

*N007303 

RA 

15 

1 

110 

(0 

077) 

-0 

02A 

(0 

08A) 

0 

196 

(0 

0A3) 

*N00730A 

RA 

16 

0 

887 

(0 

.072) 

-0 

007 

(0 

100) 

0 

.  223 

(0 

053) 

*N007305 

RA 

17 

0 

.  029 

(0 

.0A2) 

0 

.010 

(0 

.077) 

0 

.  133 

(0 

050) 

*N007306 

RA 

18 

1 

.  009 

(0 

.057) 

~0 

.  116 

(0 

.059) 

0 

.  103 

(0 

.035) 

*N013A01 

RA 

19 

1 

.  i.03 

(0 

.177) 

-0 

.  250 

(0 

.  107) 

0 

.  157 

(0 

.035) 

*N013A02 

RA 

20 

1 

.  '.38 

(0 

.  189) 

-0 

.  862 

(0 

.175) 

0 

.  205 

(0 

.0A8) 

*N013A03 

RA 

21 

1 

.  A9A 

(0 

.223) 

-0 

.  278 

(0 

.  116) 

0 

.  199 

(0 

.033) 

N021601 

R5 

7 

0 

.  619 

(0 

.022) 

-0 

.  263 

(0 

.  02A) 

0 

.  252 

(0 

.012) 

N021602 

R5 

8 

0 

.785 

(0 

.018) 

0 

.  8A6 

(0 

.039) 

0 

.  15A 

(0 

.008) 

N021603 

R5 

g 

0 

.  368 

(0 

.018) 

1 

.  238 

(0 

.085) 

0 

.216 

(0 

.009) 

N021604 

R5 

10 

1 

.  381 

(0 

.027) 

0 

.  267 

(0 

.030) 

0 

.  159 

(0 

.009) 

N021605 

R5 

11 

0 

.  79A 

(0 

.025) 

1 

.  079 

(0 

.059) 

0 

.389 

(0 

.008) 

*N003201 

R5 

12 

1 

.  20  7 

(0 

.088) 

-0 

.  593 

(0 

.087) 

0 

.  17 1 

(0 

.056) 

*N003202 

R5 

13 

1 

.  590 

(0 

.12A) 

0 

.  012 

(0 

.093) 

0 

.  227 

(0 

.038) 

*N003203 

R5 

1 A 

1 

.  215 

(0 

.  101) 

0 

.  240 

(0 

.  107) 

0 

.  222 

(0 

.039) 

*N00320A 

R5 

15 

1 

.  A  57 

(0 

.  120) 

0 

.  260 

(0 

.  112) 

0 

.  238 

(0 

.035) 

N021701 

R5 

16 

1 

.  120 

(0 

.035) 

-0 

.  A63 

(0 

.028) 

0 

.  229 

(0 

.014) 

N021702 

R5 

17 

0 

.  915 

(0 

.018) 

1 

.016 

(0 

.039) 

0 

.  108 

(0 

.007) 

N021703 

R5 

18 

1 

.351 

(0 

.026) 

1 

.  17A 

(0 

.053) 

0 

.289 

(0 

.007) 

N021201 

R6 

5 

0 

.861 

(0 

.02A) 

-0 

.  120 

(0 

.023) 

0 

.  180 

(0 

.008) 

N021202 

R6 

6 

0 

.628 

(0 

.021) 

-0 

.  153 

(0 

.022) 

0 

.  195 

(0 

.009) 

N021203 

R6 

7 

0 

.70A 

(0 

.022) 

0 

.  113 

(0 

.027) 

0 

.20A 

(0 

.008) 

N02120A 

R6 

8 

0 

.726 

(0 

.022) 

-0 

.262 

(0 

.021) 

0 

.  190 

(0 

.009) 

N021301 

R6 

9 

1 

.081 

(0 

.02A) 

-0 

.168 

(0 

.019) 

0 

.0 

(0 

.0  ) 

N021303 

R6 

11 

0 

.99A 

(0 

.031) 

~0 

.7A5 

(0 

.02A) 

0 

.  190 

(0 

.010) 

N02130A 

R6 

12 

0 

.  AA6 

(0 

.017) 

0 

.397 

(0 

.037) 

0 

.  172 

(0 

.008) 

N021305 

R6 

13 

0 

.978 

(0 

.02A) 

0 

.21A 

(0 

.028) 

0 

.  189 

(0 

.007) 

N021801 

R6 

lA 

1 

.230 

(0 

.027) 

-0 

.008 

(0 

.02A) 

0 

.0 

(0 

.0  ) 

N021803 

R6 

16 

1 

.18A 

(0 

.031) 

0 

.29A 

(0 

.038) 

0 

.291 

(0 

.010) 

N021805 

R6 

18 

0 

.965 

(0 

.027) 

-0 

.276 

(0 

.02A) 

0 

.0 

(0 

.0  ) 

*Used  In  1984  reading  scale. 
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Table  E.3 

1986  IRT  Parameters,  Reading,  Grade  11/Age  17 


FIELD 

BLOCK 

ITEM 

A 

SE 

B 

SE 

c 

SE 

*N002801 

Rl 

20 

1. 

921 

(0. 

llA  ) 

-0  . 

767 

(0. 

081) 

0 . 

175 

(0 

028) 

*N002802 

Rl 

21 

1. 

896 

(0. 

110) 

-0. 

912 

(0. 

092) 

0  . 

1A3 

(0  . 

028) 

*N002001 

Rl 

22 

1. 

197 

(0. 

065) 

-0. 

013 

(0. 

050) 

0  . 

131 

(0. 

02C) 

*N002002 

Rl 

23 

1 . 

AAA 

(0. 

08A) 

-0 . 

0A2 

(0. 

055) 

0 . 

203 

(0. 

020) 

*N002003 

Rl 

2A 

1 . 

583 

(0. 

093) 

-0. 

229 

(0. 

05A) 

0  . 

224 

(0 

022) 

*N001501 

Rl 

25 

1 . 

808 

(0. 

130) 

-1 . 

313 

(0. 

152) 

0 . 

225 

(0 

0A7) 

*N001502 

Rl 

26 

1 . 

6A3 

(0. 

098) 

-0 

507 

(0 

061) 

0 . 

182 

(0. 

026) 

*N001503 

Rl 

27 

1. 

3A5 

(0. 

088) 

-0 

902 

(0 

086) 

0 . 

207 

(0 

0A3) 

*N00150A 

Rl 

28 

1. 

AA8 

(0. 

089) 

-0 . 

650 

(0 

068) 

0 . 

173 

(0 

032) 

*N003101 

Rl 

29 

1. 

571 

(0. 

100) 

-0 

6A5 

(0. 

073) 

0 . 

267 

(0 

032) 

*N003102 

Rl 

30 

1 . 

530 

(0. 

083) 

-0 

359 

(0. 

051) 

0 . 

1 A5 

(0 

023) 

*N00310A 

Rl 

31 

0 . 

70A 

(0. 

0A2) 

1 

923 

(0. 

12A  ) 

0 . 

0 

(0 

0  ) 

*N008201 

R2 

10 

2. 

72A 

(0. 

302) 

-0 

A71 

(0. 

131) 

0 . 

323 

(0 

052) 

*N008202 

R2 

11 

1. 

1A6 

(0. 

102) 

-0 

065 

(0. 

102) 

0 . 

188 

(0 

052) 

*N008203 

R2 

12 

1 

5A3 

(0 

lAl) 

-0 

289 

(0. 

lOA) 

0 . 

2A7 

(0 

05A) 

*N00820A 

R2 

13 

2. 

600 

(0 

236) 

-0 

228 

(0 

092) 

0 

209 

(0 

038) 

*N008205 

R2 

lA 

2 

1A5 

(0 

188) 

-0 

256 

(0 

092) 

0 

205 

(0 

0A2) 

*N005001 

R2 

15 

1 

993 

(0 

102) 

1 

380 

(0 

159) 

0 

211 

(0 

Oil) 

*N005002 

R2 

16 

0 

859 

(0 

108) 

1 

288 

(0 

240) 

0 

26A 

(0 

029) 

*N005003 

R2 

17 

0 

737 

(0 

105) 

1 

905 

(0 

331) 

0 

135 

(0 

02A) 

*N003001 

R2 

18 

1 

293 

(0 

109) 

1 

153 

(0 

169) 

0 

207 

(0 

013) 

♦N003003 

R2 

19 

2 

29A 

(0 

109) 

1 

724 

(0 

190) 

0 

120 

(0 

006) 

♦N00A601 

R3 

16 

0 

899 

(0 

078) 

0 

179 

(0 

104  ) 

0 

18A 

(0 

0A8) 

*N00A602 

R3 

17 

1 

318 

(0 

103) 

-0 

085 

(0 

092) 

0 

2A  9 

(0 

OAA  ) 

♦N00A603 

R3 

18 

1 

A85 

(0 

113) 

-0 

516 

(0 

089) 

0 

226 

(0 

05A) 

*N007A01 

RA 

8 

1 

098 

(0 

076) 

0 

531 

(0 

096) 

0 

123 

(0 

027) 

*N007A02 

RA 

9 

1 

30A 

(0 

08A  ) 

-0 

317 

(0 

075) 

0 

176 

(0 

0A5) 

*N007A03 

RA 

10 

1 

756 

(0 

119) 

0 

21A 

(0 

093) 

0 

233 

(0 

027) 

*N007A0A 

RA 

11 

0 

985 

(0 

072) 

0 

060 

(0 

088) 

0 

181 

(0 

OAA) 

*N007A05 

RA 

12 

0 

887 

(0 

102) 

1 

AOl 

(0 

229) 

0 

187 

(0 

025) 

*N007301 

RA 

13 

1 

183 

(0 

091) 

-0 

39A 

(0 

100) 

0 

278 

(0 

059) 

*N007302 

RA 

lA 

0 

818 

(0 

059) 

0 

285 

(0 

08A  ) 

0 

136 

(0 

039) 

*N007303 

RA 

15 

1 

110 

(0 

077  ) 

-0 

02A 

(0 

08A) 

0 

196 

(0 

0A3) 

*N00730A 

RA 

16 

0 

887 

(0 

072) 

-0 

007 

(0 

100) 

0 

223 

(0 

053) 

*N007305 

RA 

17 

0 

529 

( 0 

0A2 ) 

0 

010 

(0 

077  ) 

0 

133 

(0 

050) 

*N007306 

RA 

18 

1 

009 

( 0 

057 ) 

-0 

116 

(0 

059) 

0 

103 

(0 

035) 

*N013A01 

RA 

19 

1 

203 

( 0 

177 ) 

-0 

250 

(0 

107  ) 

0 

157 

(0 

035) 

*N013A02 

RA 

20 

1 

A38 

(0 

189 ) 

-0 

862 

(0 

175) 

0 

205 

(0 

0A8) 

*N013A03 

RA 

21 

1 

A9A 

(0 

223 ) 

-0 

278 

(0 

116) 

0 

199 

(0 

033) 

N021601 

R5 

7 

0 

619 

(0 

022) 

-0 

263 

(0 

02A  ) 

0 

252 

(0 

012) 

N021602 

R5 

8 

0 

785 

(0 

018 ) 

0 

8A6 

(0 

039) 

0 

15A 

(0 

008) 

N021603 

R5 

9 

c 

368 

(0 

018) 

1 

.238 

(0 

085) 

0 

216 

(0 

009) 

N02160A 

R5 

10 

1 

.381 

( 0 

027  ) 

0 

.267 

(0 

030  ) 

0 

159 

(0 

.009) 

N021605 

R5 

11 

0 

.79A 

( 0 

.  025) 

1 

.079 

(0 

059) 

0 

389 

(0 

.008) 

*N003201 

R5 

12 

1 

.207 

(0 

.  088 ) 

-0 

.593 

(0 

087  ) 

0 

171 

(0 

.056) 

*N003202 

R5 

13 

1 

.590 

(0 

12A  ) 

0 

.012 

(0 

.093) 

0 

227 

(0 

.038) 

*N003203 

R5 

lA 

1 

.215 

(0 

.  101) 

0 

.2A0 

(0 

107) 

0 

222 

(0 

.039) 

*N00320A 

R5 

15 

1 

.  A57 

(0 

.  120) 

0 

.260 

(0 

112) 

0 

238 

(0 

.035) 

N021701 

R5 

16 

1 

.  120 

( 0 

.  035 ) 

-0 

.  A63 

( 0 

028 ) 

0 

.229 

(0 

.  OlA  ) 

N021702 

R5 

17 

0 

.915 

(0 

.018) 

1 

.016 

(0 

.039) 

0 

.  108 

(0 

.007) 

N021703 

R5 

18 

1 

.351 

(0 

.026) 

1 

.  17A 

(0 

.053) 

0 

.289 

(0 

.007  ) 

N021201 

R6 

5 

0 

.861 

(0 

.02A) 

-0 

.  120 

(0 

.023) 

0 

.  180 

(0 

.008) 

N021202 

R6 

6 

0 

.628 

(0 

.021) 

-0 

.  153 

(0 

.022) 

0 

.  195 

(0 

.009) 

N021203 

R6 

7 

0 

.70A 

(0 

.022) 

0 

.113 

(0 

.027) 

0 

.20A 

(0 

.008) 

N02120A 

R6 

8 

0 

.726 

(0 

.022) 

-0 

.262 

(0 

.021) 

0 

.  190 

(0 

.009) 

N021301 

R6 

9 

1 

.081 

(0 

.02A) 

-0 

.  168 

(0 

.019) 

0 

.0 

(0 

.0  ) 

N021303 

R6 

11 

0 

.99A 

(0 

.031) 

-0 

.7A5 

(0 

.  02A) 

0 

.  190 

(0 

.010) 

N02130A 

R6 

12 

0 

.  AA6 

(0 

.017) 

0 

.397 

(0 

.037) 

0 

.  172 

(0 

.008) 

N021305 

R6 

13 

0 

.978 

(0 

.  02A) 

0 

.21A 

(0 

.028) 

0 

.  189 

(0 

.007) 

N021801 

R6 

lA 

1 

.230 

(0 

.027) 

-0 

.008 

(0 

.02A) 

0 

.0 

(0 

.0  ) 

N021803 

R6 

16 

1 

.18A 

(0 

.031) 

0 

.29A 

(0 

.038) 

0 

.291 

(0 

.010) 

N021805 

R6 

18 

0 

.965 

(0 

.027) 

-0 

.276 

(0 

.02A) 

0 

.0 

(0 

.0  ) 

*Used  in  198/i  reading  scale. 
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610 


Tab!©  E.4 

1986  IRT  Parameters,  Mathematics,  Grade  3/Age  9 
Measurement  Subscale 


PTFT  n 

ITEM 

SE 

lI9R7Rn  1 

Ml 

3 

2 . 

096 

(0. 

054) 

H9  K7  Rn  9 

Ml 

18 

2 . 

24  6 

(0. 

086) 

Ml 

21 

202 

(0. 

438) 

N9  RR 1 n 1 

Do  X u  X 

Ml 

22 

275 

(0. 

035) 

riZDa  XU  X 

Ml 

nx 

9 

ZO 

930 

(0. 

095) 

1J9  Rfi  9n  1 

nzDozu  X 

Ml 

nx 

Z** 

X  . 

Rn  7 

(0. 

069) 

1J9  ^9 1  n  1 
nzDZ  xu  X 

Ml 

n  X 

9  S 
z  o 

Q 

832 

(0. 

052) 

M9  S9  Rn 1 
nZDZ DU  X 

Ml 

nx 

26 

2 

112 

(0. 

031) 

nZ / DDU  X 

M9 

nz 

z 

X 

Q  A  7 

(0. 

044) 

V  9  ^  1  A  n  1 
nZDXHU  X 

M9 

nz 

XD 

X 

*l  /  z 

(0 

044) 

M9  S9  nn 1 

n ZDZ U U  X 

M2 

25 

\ 

590 

(0 

075) 

n  ^  u  /  w  w  X 

M3 

16 

283 

(0 

041) 

N25160 1 

MA 

13 

1 

105 

(0 

028) 

N265201 

M4 

14 

1 

463 

(0 

068) 

N252901 

MA 

23 

1 

815 

(0 

071) 

N237701 

M5 

23 

1 

563 

(0 

172) 

N219101 

M6 

11 

1 

.870 

(0 

043) 

N204901 

M6 

12 

0 

.924 

(0 

024) 

N20A601 

M6 

13 

1 

.087 

(0 

.027) 

N216501 

M6 

17 

0 

.483 

(0 

.030) 

N236401 

M6 

19 

1 

.  164 

(0 

.337) 

N216601 

M6 

22 

1 

.428 

(0 

.049) 

N231701 

M6 

25 

2 

.383 

(0 

.097) 

N215301 

M6 

26 

2 

.788 

(0 

.064) 

N236S01 

M7 

11 

1 

.114 

(0 

.032) 

H215401 

M7 

14 

1 

.425 

(0 

.048) 

B  SE  C  SE 


-0 . 

894 

(0 . 

052 ) 

0 . 

257 

(0  . 

017 ) 

-0 . 

592 

(0 . 

058 ) 

0 . 

227 

( 0 . 

015) 

1 . 

070 

(0 . 

447 ) 

0 . 

282 

(0 ! 

012 ) 

0 . 

576 

(  0 

031 ) 

0 . 

294 

( 0 . 

008 ) 

Q  ^ 

329 

(  0 

040 ) 

0 . 

220 

(  0 

008 ) 

-Q  ^ 

256 

( 0 . 

026 ) 

0 . 

120 

(Q 
\  *J  • 

Oil) 

0 . 

880 

(  0 

064 ) 

0 

197 

(  0 

009 ) 

445 

(  0 

049 ) 

0 . 

210 

(  0 

006 ) 

025 

(  0 

047 ) 

0 . 

211 

(Q 

019) 

-Q 

7  83 

( 0 

041 ) 

0 

150 

(0 

016) 

0 

993 

(0 

075) 

0 

228 

(6 

008) 

-1 

194 

(0 

055) 

0 

230 

(0 

024) 

-1 

154 

(0 

040) 

0 

266 

(0 

016) 

-0 

430 

(0 

041) 

0 

456 

(0 

010) 

0 

436 

(0 

036) 

0 

114 

(0 

005) 

0 

.055 

(0 

067) 

0 

262 

(0 

.011) 

-0 

.599 

(0 

.026) 

0 

.  160 

(0 

.010) 

-0 

.645 

(0 

.023) 

0 

.  139 

(0 

.010) 

-0 

.828 

(0 

.028) 

0 

.212 

(0 

.012) 

1 

.078 

(0 

.071) 

0 

.248 

(0 

.007  ) 

1 

.007 

(0 

.325) 

0 

.285 

(0 

.008) 

-0 

.104 

(0 

.021) 

0 

.  151 

(0 

.008) 

0 

.239 

(0 

.036) 

0 

.289 

(0 

.007) 

0 

.592 

(0 

.038) 

0 

.087 

(0 

.005) 

-1 

.688 

(0 

.060) 

0 

.223 

(0 

.028) 

-0 

.686 

(0 

.043) 

0 

.213 

CO 

.015) 

-57*- 


Table  E.5 

1986  IRT  Pi  ameters,  Mathematics,  Grade  3/Age  9 
Numbers  and  Operations — High  Level  Subscale 


FIELD 

BLOCK 

ITEM 

SE 

2 

SE 

p 

SE 

N277401 

Ml 

2 

0 

736 

(0 

024) 

-2 

216 

(0 

077) 

n 
u 

1  S7 

(0 

02S) 

N276101 

Ml 

12 

1 

607 

(0 

033) 

~0 

814 

(0 

033) 

n 
u 

n 
u 

(0 

0  ) 

N272301 

M2 

1 

1 

555 

(0 

026) 

- 

674 

(0 

044) 

n 
u 

(0 

024) 

N27S401 

M2 

7 

1 

414 

(0 

02S) 

"0 

916 

(0 

028) 

n 
u 

n 
u 

(0 

0  ) 

N2S9101 

M4 

16 

40  9 

(0 

OSS) 

u 

H  9  o 

(0 

040) 

r\ 

U 

O  7  c 

(0 

Oil) 

N276501 

M4 

18 

o 

n  sft 

U  J  o 

(0 

043) 

U 

/Z  o 

(0 

037) 

0 

171 

(0 

.010) 

Cl  ^  w  O  U  U  A 

MS 

15 

1  n  R 

1 U  D 

(0 

042) 

U 

DU  9 

(0 

039) 

0 

232 

(0 

013) 

1  ft 

1 

(0 

061) 

~0 

S4  1 

(0 

044) 

0 

231 

(0 

012) 

MS 

1  Q 

1 

(0 

048) 

~0 

623 

(0 

041) 

0 

206 

(0 

013) 

u 

83  S 

(0 

081) 

0 

SSO 

(0 

069) 

0 

188 

(0 

009) 

N235201 

MS 

22 

1 

796 

(0 

124) 

-0 

058 

(0 

04S) 

0 

143 

(0 

009) 

N238201 

MS 

26 

2 

129 

(0 

132) 

-0 

223 

(0 

0S3) 

0 

170 

(0 

010) 

N240001 

MS 

27 

2 

336 

(0 

209) 

-0 

OSS 

(0 

0S8) 

0 

139 

(0 

.009) 

N230501 

M6 

9 

1 

709 

(0 

.070) 

-0 

092 

(0 

023) 

0 

280 

(0 

.007) 

N230101 

M6 

14 

0 

811 

(0 

024) 

-0 

728 

(0 

027) 

0 

220 

(0 

Oil) 

N217201 

M6 

16 

1 

S21 

(0 

0S7) 

-0 

1S8 

(0 

021) 

0 

244 

(0 

008) 

N207401 

M6 

18 

1 

433 

(0 

0S9) 

-0 

101 

(0 

022) 

0 

290 

(0 

008) 

N239101 

M7 

12 

1 

066 

(0 

.026) 

-1 

S99 

(0 

.OSO) 

0 

198 

(0 

023) 

H206601 

M7 

16 

0 

704 

(0 

.OlS) 

0 

012 

(0 

.011) 

0 

07S 

(0 

.007) 

N207801 

M7 

17 

1 

73S 

(0 

.021) 

0 

.779 

(0 

.021) 

0 

213 

(0 

.004) 

N237601 

M7 

18 

1 

399 

(0 

.  103) 

-0 

097 

(0 

.048) 

0 

280 

(0 

.010) 

N234101 

M7 

21 

0 

931 

(0 

.022) 

0 

.211 

(0 

.016) 

0 

1S2 

(0 

.007) 

N239601 

M7 

28 

1 

9S9 

(0 

.292) 

1 

.4S0 

(0 

.364) 

0 

117 

(0 

.006) 

-S7S- 


Table  E.6 

1986  XRX  Parameters,  Mathematics,  Grade  3/Age  9 
Numbers  and  Operations—Knowledge  and  Skills  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N276801 

Ml 

4 

0. 

503 

0 19 ) 

N276802 

Ml 

5 

0. 

769 

( 0 

017  ) 

N276803 

Ml 

6 

0. 

8A1 

(0 . 

018 ) 

N257201 

Ml 

11 

1. 

090 

(  0 

0  A5 ) 

N286101 

Ml 

13 

0. 

855 

023 ) 

N272102 

Ml 

15 

1 

150 

(  0 

065 ) 

N28^.001 

Ml 

16 

1 

162 

(  0 

022) 

N28A002 

Ml 

17 

1 

738 

f  n 

028 ) 

N257801 

M2 

3 

0 

89A 

f  fi 
K  u 

N273501 

M2 

6 

0 

898 

n  A  7  ^ 

N277501 

M2 

8 

0 

886 

N277601 

M2 

9 

1 

285 

021 ) 

N277602 

M2 

10 

1 

300 

t  n 

n  9  ^  ^ 

N277603 

M2 

11 

1 

23A 

t  n 

no^  ^ 

\j£.0  ) 

N261A01 

M2 

12 

0 

620 

\J  0  /  ) 

N276001 

M2 

21 

0 

9A2 

022  ) 

N276002 

M2 

22 

1 

202 

(0 

059) 

N286102 

M2 

23 

0 

.  865 

(0 

023) 

N272801 

M3 

15 

1 

.  058 

(0 

038) 

N272101 

M3 

17 

1 

.  101 

(0 

053) 

N277901 

MA 

8 

0 

.  899 

(0 

.OlA) 

N277902 

m 

9 

0 

.  893 

(0 

.OlA) 

N277903 

m 

10 

1 

.017 

(0 

.016) 

N272302 

MA 

11 

1 

.  185 

(0 

.027) 

N272601 

m 

17 

1 

.  096 

(0 

.025) 

N282901 

m 

20 

0 

.  998 

(0 

.02A) 

N257701 

m 

22 

1 

.  766 

(0 

.296) 

N235601 

M6 

21 

1 

.  1A9 

(0 

.03A) 

N202801 

M7 

15 

0 

.955 

(0 

.OAA  ) 

N239501 

M7 

23 

0 

.97A 

(0 

.051) 

B  SE  C  SE 


-3. 

780 

(0. 

143) 

0. 

0 

(0. 

0  ) 

-2. 

066 

(0. 

049) 

0. 

0 

(0. 

0  ) 

-0 . 

642 

(0, 

019) 

0. 

0 

(0. 

0  ) 

-0 . 

724 

(0. 

044  ) 

0  . 

238 

(0. 

015) 

-0. 

860 

(0. 

032) 

0, 

0 

(0 

0  ) 

•0. 

295 

(0 

040) 

0. 

208 

(0 

013) 

-0. 

848 

(0 

024) 

0. 

0 

(0 

0  ) 

0. 

125 

(0 

018) 

0. 

0 

(0 

0  ) 

-0 

960 

(0 

044  ) 

0 

280 

(0 

014) 

-0 

716 

(0 

050) 

0 

352 

(0 

015) 

-0 

900 

(0 

028) 

0 

0 

(0 

0  ) 

-1 

101 

(0 

025) 

0 

0 

(0 

0  ) 

-0 

445 

(0 

017) 

0 

0 

(0 

0  ) 

-0 

554 

(0 

018) 

0 

0 

(0 

0  ) 

-0 

256 

(0 

032) 

0 

225 

(0 

013) 

-1 

273 

(0 

039) 

0 

0 

(0 

0  ) 

0 

299 

(0 

034  ) 

0 

0 

(0 

0  ) 

-0 

047 

(0 

014) 

0 

0 

(0 

0  ) 

-1 

152 

(0 

.053) 

0 

198 

(0 

020) 

-0 

817 

(0 

.055) 

0 

257 

(0 

.018) 

-1 

871 

(0 

.034  ) 

0 

0 

(0 

.0  ) 

-1 

839 

(0 

.033) 

0 

0 

(0 

.0  ) 

-1 

042 

(0 

.021) 

0 

0 

(0 

.0  ) 

-1 

.074 

(0 

.034  ) 

0 

.232 

(0 

.012) 

-0 

.376 

(0 

.020) 

0 

.0 

(0 

.0  ) 

-0 

.484 

(0 

.021) 

0 

.0 

(0 

.0  ) 

1 

.115 

(0 

.248) 

0 

.  197 

(0 

.006) 

0 

.365 

(0 

.021) 

0 

.  164 

(0 

.006) 

-0 

.  544 

(0 

.  040) 

0 

.247 

(0 

.013) 

-0 

.454 

(0 

.042) 

0 

.243 

(0 

.013) 

-576- 
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Table  E.7 

1986  IRT  Parameters,  Mathematics,  Grade  7/Age  13 
Measurement  Subscale 


FIELD 

BLOCK 

ITEM 

N267201 

Ml 

23 

N265202 

Ml 

30 

1 

N266801 

Ml 

31 

0 

N252901 

Ml 

32 

1 

N265201 

Ml 

36 

1 

N26S901 

Ml 

40 

1 

N2S2101 

Ml 

41 

0 

N269001 

Ml 

44 

1 

H269101 

M2 

26 

1 

N261801 

M2 

35 

0 

N252001 

M.» 

40 

1 

N26g201 

M2 

44 

1 

N266101 

M3 

27 

1 

N265902 

M3 

31 

2 

N285201 

m 

29 

1 

N266701 

MA 

32 

0 

N251201 

M5 

26 

0 

N28<»A01 

M5 

27 

1 

N252201 

M5 

30 

1 

N265903 

M5 

31 

N251801 

M5 

32 

1 

N266001 

M5 

38 

1 

N252601 

M5 

AO 

2 

N267901 

M5 

41 

1 

N219101 

M6 

15 

1 

N204901 

M6 

16 

0 

N204601 

M6 

17 

1 

N216501 

M6 

21 

0 

N216601 

M6 

26 

1 

N231701 

M6 

29 

2 

N215301 

M6 

32 

2 

N251901 

M7 

26 

1 

N26A501 

M7 

29 

1 

N232901 

M7 

33 

1 

N268201 

M8 

16 

1 

N215701 

M8 

2A 

1 

N231501 

M8 

30 

0 

N215601 

M8 

36 

1 

N231801 

M8 

38 

1 

N218S01 

M8 

A7 

1 

N217101 

M8 

51 

2 

N252701 

M8 

55 

1 

N264301 

MS 

58 

1 

N216401 

M9 

49 

2 

N216301 

M9 

51 

1 

A  SE  B 


063 

(0 

.  080 ) 

-0 

.  289 

260 

(0 

.076) 

0 

.  235 

800 

(0 

.  030 ) 

-0 

.  188 

815 

(0 

.  071 ) 

0 

.  A36 

A63 

(0 

.068) 

-0 

.  A30 

643 

(0 

.033) 

1 

.  088 

832 

(0 

.052) 

0 

.  880 

A68 

(0 

.033) 

0 

.  521 

930 

(0 

.  095) 

0 

329 

915 

(0 

03A  ) 

-0 

2A5 

590 

(0 

075) 

0 

993 

693 

(0 

023) 

1 

186 

275 

(0 

035) 

0 

576 

227 

(0 

OAl) 

1 

416 

291 

(0 

032) 

0 

205 

820 

(0 

023) 

1 

A21 

781 

(0 

021 ) 

0 

993 

028 

(0 

023 ) 

0 

030 

331 

f  A 

Oz3 ) 

0 

802 

886 

(0 

025) 

1 

050 

llA 

(0 

022) 

0 

819 

A75 

(0 

020) 

1 

713 

112 

(0 

031 ) 

1 

AA5 

078 

(0 

020 ) 

2 

105 

870 

(0 

0A3) 

-0 

599 

g2A 

(0 

02A  ) 

-0 

6A5 

087 

(0 

027) 

-0 

828 

A83 

(0 

030 ) 

1 

078 

A28 

(0 

0A9) 

-0 

lOA 

383 

(0 

097) 

0 

239 

788 

(0 

06A) 

0 

592 

AA9 

(0 

065) 

0 

221 

013 

(0 

0A6) 

0 

115 

795 

(0 

026) 

0 

760 

607 

(0 

069) 

-0 

256 

AOO 

(0 

029) 

0 

566 

699 

<o 

020) 

0 

355 

628 

(0 

028) 

0 

896 

296 

(0 

031) 

1 

085 

811 

(0 

028) 

1 

110 

307 

(0 

026) 

1 

AA2 

A37 

(0 

021) 

1 

361 

A88 

(0 

022) 

1 

506 

Oil 

(0 

030) 

0 

778 

538 

(0 

023) 

1 

A05 

SE  C  SE 


(0 

039 ) 

u 

9  R  1 

C  U 

.  UZU  ) 

( 0 

{J  ) 

u 

O  A  7 

.  Z  A  / 

( 0 

.016) 

(0 

023 ) 

Q 

.  205 

.  U  lO  ^ 

(0 

036) 

Q 

1 1  A 

.  UU  3  ^ 

(0 

OAl) 

Q 

.4  56 

n  1  n  ^ 

(0 

0A2) 

0 

.297 

( 0 

n  n  ft  \ 

(0 

06A  ) 

0 

197 

( 0 

009 ) 

(0 

026) 

0 

115 

( 0 

010) 

(0 

040 ) 

0 

.  220 

( 0 

008 ) 

(0 

02A  ) 

Q 

195 

n  1  7  ^ 

(0 

075) 

0 

.  228 

( 0 

008) 

(0 

029 ) 

0 

0 

(  0 

w  } 

(0 

031 ) 

0 

294 

( 0 

008 ) 

(0 

064  ) 

0 

399 

( 0 

007) 

(0 

018 ) 

0 

0 

( 0 

u  } 

(0 

0A7 ) 

0 

210 

/  0 

009) 

(0 

035) 

0 

162 

( 0 

0 1  n  ^ 

( 0 

015) 

Q 

n  ^ 

U  I 

(0 

02A  ) 

0 

0 

(  0 

u  } 

(0 

039 ) 

0 

257 

(  0 

009) 

(0 

02A  ) 

0 

0 

(  0 

(0 

036 ) 

0 

Q 

( 0 

(0 

0A9 ) 

0 

210 

( 0 

006) 

(0 

0  A  9 ) 

0 

0 

( 0 

(0 

026 ) 

0 

160 

( 0 

010) 

(0 

023 ) 

0 

139 

(0 

028 ) 

0 

212 

(  0 

0 12  ) 

(0 

071 ) 

0 

248 

( 0 

00  7) 

(0 

021) 

0 

151 

( 0 

008 ) 

(0 

036) 

0 

289 

(0 

007) 

(0 

038) 

0 

087 

(0 

005) 

(0. 

025) 

0 

149 

(0 

010) 

(0. 

017) 

0 

0 

(0 

0  ) 

(0. 

02A) 

0 

150 

(0 

006) 

(0. 

026) 

0 

120 

(0 

Oil) 

(0. 

02A) 

0 

220 

(0 

008) 

(0. 

019) 

0 

233 

(0 

010) 

(0. 

031) 

0 

270 

(0. 

007) 

(0. 

0A2) 

0 

411 

(0. 

007) 

(0. 

037) 

0 

344 

(0. 

007) 

(0. 

OAl) 

0 

175 

(0. 

005) 

(0. 

032) 

0 

0 

(0. 

0  ) 

(0. 

035) 

0 

0 

(0. 

0  ) 

(0. 

029) 

0 

156 

(0. 

008) 

(0. 

039) 

0 

284 

(0. 

006) 
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Table  E.8 

1986  IRT  Parameters,  Mathematics,  Grade  7/Age  13 
Numbers  and  Operations--Hifth  Level  Subscale 


FIELD      BLOCK  ITEM        A  SE  B  SE  C  SE 


N286201 

Ml 

2A 

0.98A 

(0.03A) 

-0.095 

(0.022) 

0. 

203 

(0.015) 

N2S8801 

Ml 

38 

2.10A 

(0.032) 

1.  170 

(0.0A2) 

0. 

326 

(0.007) 

N275001 

Ml 

A2 

1  .218 

(0.0A9) 

0.817 

(O.0A2) 

0. 

0 

(0.0  ) 

N2&6301 

Ml 

A5 

2.002 

(0.032) 

0.758 

(0.031) 

0. 

197 

(0.009) 

N277A01 

M2 

8 

0.736 

(0.02A) 

-2.216 

(0.077) 

0. 

157 

(0.025) 

N282201 

M2 

28 

1.53A 

(0.026) 

1.  OAA 

(0.03A) 

0. 

26A 

(0.008) 

N258802 

M2 

31 

2.279 

(0.03A) 

0.808 

(0.033) 

0. 

215 

(0.008) 

N261501 

M2 

3A 

1.081 

(0.038) 

-0.506 

(0.028) 

0. 

130 

(0.019) 

N261601 

M2 

36 

0.808 

(0.020) 

1.393 

(O.OAl) 

0. 

222 

(0.008) 

H261301 

M2 

37 

0.9A0 

(0.022) 

0.272 

(0.019) 

0. 

109 

(0.012) 

N261201 

M2 

38 

0.935 

(0  .02A) 

0.21A 

(0.020) 

0. 

150 

(0.013) 

N281401 

M2 

39 

1.110 

(0.020) 

0.890 

(0.025) 

0. 

107 

(0  .009) 

N256501 

M3 

30 

2.150 

(0.038) 

0.890 

(0  .OAl J 

0. 

318 

(0.009) 

N263901 

MA 

30 

1.58A 

(0.026) 

0.726 

(0.026) 

0. 

153 

(0.008) 

N255302 

M4 

37 

1.809 

(0.026) 

1.AA3 

(0 .0A2) 

0. 

211 

(0. 007 ) 

N258201 

MA 

39 

1.065 

(0  021) 

0.618 

(0.022) 

0. 

0 

(0.0  ) 

N260902 

MA 

AO 

2.739 

(0  .OAl) 

1.288 

(0.057) 

0  . 

289 

(0.009) 

N285A01 

MA 

Al 

1.378 

(0  .021) 

1.808 

(0.039) 

0. 

0 

(0.0  ) 

N257901 

M5 

23 

2.217 

(0  .05A) 

0  .512 

(0 .OAO) 

0. 

3A0 

(0.012) 

K279301 

M5 

3A 

1.357 

(0.012) 

1.573 

(0.021) 

0. 

0 

(0.0  ) 

N259501 

M5 

36 

1.292 

(0.019) 

0  .972 

(0.02A) 

0. 

0 

(0.0  ) 

N230501 

M6 

13 

1.709 

(0.070) 

-0.092 

(0.023) 

0. 

280 

(0.007) 

N230101 

M6 

18 

0.811 

(0  .02A) 

-0.728 

(0.027) 

0 

220 

(0.011) 

N217201 

M6 

20 

1.521 

(0.057) 

-0.158 

(0.021) 

0 

2AA 

(0. 008) 

N207A01 

M6 

22 

1.  A33 

(0.059) 

-0.101 

(0.022) 

0 

290 

(0.008) 

N207601 

M6 

33 

1.660 

(0.050) 

1.116 

(0. 05A ) 

0 

176 

(0.007) 

N208A01 

M6 

3A 

1.  589 

(0.068) 

0.318 

(0.033) 

0 

236 

(0.011) 

N201101 

M6 

A2 

1 .  A6A 

(0.021) 

0.701 

(0.022) 

0 

.158 

(0. 008) 

N20A101 

M7 

18 

1.850 

(0  .0A6) 

-0.012 

(0.019) 

0 

.  17A 

(0.010) 

N205201 

M7 

22 

1.  A53 

(0,023) 

0  .531 

(0.019) 

0 

.  188 

(0.007) 

N207801 

M7 

2A 

1.  735 

(0.021) 

0.779 

(0.021) 

0 

.213 

(0. OOA  ) 

N206601 

M7 

28 

0.  70A 

(0.015) 

0  .012 

(0.011) 

0 

.075 

(0 . 007 ) 

N259901 

M7 

3A 

1.  7A2 

(0 .025) 

0.801 

(0.027) 

0 

.199 

(0  .007  ) 

N202501 

M7 

38 

1.296 

(0.018) 

1.036 

(0.02A) 

0 

.197 

(0. 006) 

N201A01 

M7 

39 

1.010 

(0.015) 

0.985 

(0.021) 

0 

.071 

(0  .  006) 

N201A02 

M7 

AO 

0.  993 

(0  .016) 

0  .601 

(0 . 017 ) 

0 

.079 

( 0 . 008) 

N205901 

M7 

A2 

1.529 

(0.020) 

1 .326 

(0.032) 

0 

.267 

(0  .  006) 

N200901 

M7 

AA 

1.829 

(0.021) 

1.071 

(0.027) 

0 

.  192 

(0.006) 

N206801 

M8 

19 

2. 1A3 

(0.031) 

0.596 

(0.025) 

0 

.  192 

(0.007) 

N234201 

M8 

20 

1.  792 

(0.022) 

0  .797 

(0.022) 

0 

.175 

(0.006) 

N206S01 

M8 

27 

i.071 

(0.023) 

0  .2A6 

(0.017) 

0 

.  193 

(0.010) 

N20S101 

M8 

31 

2.  690 

(0.033) 

0  .922 

(0.035) 

0 

.376 

(0.006) 

N203801 

M8 

37 

0.879 

(0.022) 

1 .68A 

(0.051) 

0 

.339 

(0.006) 

N206701 

M8 

A6 

1.363 

(0.020) 

1.0A3 

(0.027) 

0 

.225 

(0.007) 

N260901 

M8 

5A 

3. 161 

(0.037) 

1.170 

(0.0A6) 

0 

.  182 

(0.007) 

N207101 

M9 

36 

1.822 

(0.025) 

1.0A2 

(0.031) 

0 

.3A8 

(0.006) 

N23^101 

M9 

37 

0.931 

(0.022) 

0  .211 

(0.016) 

0 

.  152 

(0.007) 

N208101 

M9 

Al 

1 .390 

(0.025) 

0.A2A 

(0.020) 

0 

.163 

(0.009) 

N271301 

M9 

AA 

2.  1A7 

(0.031) 

1 .2A9 

(0. OAA  ) 

0 

.303 

(0.007) 

N201701 

M9 

A5 

1.  A32 

(0.023) 

0.897 

(0.028) 

0 

.306 

(0.007) 
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TabLtt  E.9 

1986  IRT  Parameters,  Mathematics,  Grade  7/A8«  13 
Numbers  and  0peration8--KnowL«dg«  and  Skills  Subscal* 


t  IhLU 

BLOCK 

ITEM 

N28 190 1 

Ml 

IS 

1 

N27680 1 

Ml 

17 

0 

N276802 

Ml 

18 

0 

N276803 

Ml 

19 

0 

N27760 1 

Ml 

20 

1 

N277602 

Ml 

21 

1 

N277603 

Ml 

22 

1 

N27480 1 

Ml 

29 

1 

N25760 1 

Ml 

35 

1 

N260 10 1 

Ml 

A3 

2 

N26100 1 

Ml 

A7 

1 

N283 10 1 

Ml 

51 

2 

N27790 1 

M2 

9 

0 

N277902 

M2 

10 

0 

N27  7903 

M2 

11 

1 

N28660 1 

M2 

23 

1 

N286602 

M2 

2A 

1 

N286603 

M2 

25 

1 

N2S8803 

M2 

Al 

1 

N275301 

M3 

25 

0 

N282202 

M3 

26 

1 

N2S6301 

M4 

19 

1 

N280601 

MA 

23 

1 

N280602 

M4 

2A 

1 

N280603 

M4 

25 

1 

N280604 

M4 

26 

1 

\i  o  o  n  c  n  c 

N280605 

M4 

27 

2 

u  o  o  n  c  n  c 

ItZo  0  bO  D 

MA 

28 

1 

ItZ/  1401 

MA 

33 

1 

It  Z/  OOO  1 

MA 

35 

1 

IJ  O  7  Q  Q  n  *> 

ItZ  /  ooUZ 

36 

1 

ItZo  (» lU  1 

M5 

18 

0 

fiZo  *t  lUZ 

M5 

19 

0 

wo  Q  A  ^  n 1 

M5 

20 

0 

HO  fl  A  ^  n  o 
n  zo  *t  D  u  ^ 

M5 

O  1 
Z  1 

1 

HO fl  A  ^ n Q 
n  zo  *t  0  u  o 

M5 

o  o 
zz 

1 

HO  7  'I  Q  n  o 

no 

o  ^ 
Z  J 

1 

HO  fl     n 1 
n  zo  Du  u 1 

M  J 

o  n 
zo 

1 

HO  7  A  fl  no 
n  z / ^  o  uz 

M  J 

O  Q 

z» 

2 

HO  R  n  7  n  1 

M  J 

1 

H'^'i  ^  P  n  1 

n       D  D  U  1. 

nD 

o  ^ 
Z  0 

1 

HO  ^  fi  fi  n  /i 
nz  D  o  o  u  *t 

M7 

o  n 
z  u 

1 

HO  R  n  R  n 1 

n  ZD  U  D  U  1 

M7 

O  1 
Z  1 

1 

L%       mJ  /  "f  U  2. 

M7 

23 

N274101 

M7 

25 

2 

N278501 

M7 

30 

3 

N278502 

M7 

31 

2 

N278503 

M7 

32 

2 

N286102 

M8 

17 

0 

N225901 

M8 

23 

1 

N287301 

M8 

25 

1 

N287302 

M8 

26 

1 

N280401 

M8 

33 

1 

N230201 

M8 

35 

0 

N278905 

M8 

52 

2 

N201001 

M8 

57 

3 

A  SE  B 


335 

(0 

093) 

-0 

781 

503 

CO 

019) 

-3 

780 

769 

CO 

017) 

-2 

066 

8A1 

CO 

018) 

-0 

6A2 

288 

CO 

021) 

-1 

101 

300 

CO 

025) 

-0 

AA5 

23A 

CO 

023) 

-0 

55A 

291 

CO 

031) 

0 

A78 

552 

CO 

060) 

-0 

120 

A38 

CO 

035) 

0 

630 

326 

CO 

026) 

0 

555 

071 

CO 

025) 

1 

07'» 

899 

CO 

OlA) 

-1 

871 

893 

CO 

OlA) 

-1 

839 

017 

CO 

016) 

-1 

0A2 

538 

CO 

023) 

0 

A38 

311 

CO 

020) 

0 

538 

935 

CO 

021) 

0 

787 

7A7 

CO 

022) 

1 

030 

768 

CO 

050) 

-0 

3A5 

083 

CO 

02A) 

0 

333 

126 

CO 

019) 

0 

237 

759 

CO 

018) 

0 

67A 

935 

CO 

020) 

0 

66A 

682 

CO 

025) 

0 

2AA 

913 

CO 

017) 

0 

912 

018 

CO 

022) 

0 

537 

61A 

CO 

016) 

0 

973 

815 

CO 

029) 

1 

218 

AA5 

CO 

019) 

0 

603 

7A3 

CO 

017) 

1 

106 

670 

CO 

035) 

~2 

177 

596 

CO 

021 ) 

-0 

776 

798 

CO 

015) 

0 

156 

159 

C  0 

026 ) 

-0 

361 

59A 

C  0 

022 ) 

0 

301 

991 

CO 

028 ) 

0 

622 

331 

(  0 

021 ) 

0 

528 

350 

C  0 

031) 

0 

637 

693 

C  0 

020 ) 

0 

869 

1A9 

CO 

03A ) 

0 

365 

268 

C  0 

030 ) 

0 

200 

283 

CO 

019) 

0 

AA2 

371 

CO 

039) 

0 

353 

392 

CO 

055) 

0 

861 

117 

CO 

031) 

0 

A29 

6A7 

CO 

02A) 

0 

525 

692 

CO 

032) 

0 

337 

865 

CO 

023) 

-0 

0A7 

676 

CO 

021) 

0 

909 

195 

CO 

017) 

0 

803 

572 

CO 

019) 

1 

7A1 

OCA 

CO 

019) 

0 

235 

91A 

CO 

017) 

1 

165 

237 

CO 

033) 

1 

A67 

8A9 

CO 

0A6) 

1 

335 

SE  C  SE 


CO 

068) 

0 

263 

CO 

029) 

CO 

1A3) 

0 

0 

CO 

0  ) 

CO 

0A9) 

0 

0 

CO 

0  ) 

CO 

019) 

0 

0 

CO 

0  ) 

CO 

025) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

018) 

0 

0 

CO 

0  ) 

CO 

027) 

0 

A19 

CO 

010) 

CO 

026) 

0 

0 

CO 

0  ) 

CO 

028) 

0 

237 

CO 

009) 

CO 

02A) 

0 

236 

CO 

oil) 

CO 

030) 

0 

183 

CO 

007) 

CO 

03A) 

0 

0 

CO 

0  ) 

CO 

033) 

0 

0 

CO 

0  ) 

CO 

021) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

019) 

0 

0 

CO 

0  ) 

CO 

026) 

0 

167 

CO 

007) 

CO 

037) 

0 

222 

CO 

022) 

CO 

020) 

0 

310 

CO 

oil) 

CO 

OlA) 

0 

0 

CO 

0  ) 

CO 

016) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

016) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

OAO) 

0 

A05 

CO 

008) 

CO 

017) 

0 

0 

CO 

0  ) 

CO 

020 ) 

0 

0 

CO 

0  ) 

CO 

119) 

0 

0 

CO 

0  ) 

CO 

032) 

0 

0 

CO 

0  ) 

CO 

012) 

0 

0 

CO 

0  ) 

C  0 

018 ) 

0 

0 

CO 

0  ) 

C  0 

015) 

0 

0 

CO 

0  ) 

C  0 

02A ) 

0 

1A5 

CO 

009) 

CO 

018) 

0 

0 

CO 

0  ) 

C  0 

026) 

0 

161 

CO 

009) 

CO 

020 ) 

0 

0 

C  0 

0  ) 

CO 

021) 

0 

16A 

CO 

006) 

CO 

020 ) 

0 

376 

CO 

009) 

CO 

OlA) 

0 

0 

CO 

0  ) 

CO 

025) 

0 

260 

CO 

010) 

CO 

0A9) 

0 

512 

CO 

007) 

(0 

019) 

0 

0 

CO 

0  ) 

CO 

017) 

0 

0 

(0 

0  ) 

CO 

018) 

0 

0 

CO 

0  ) 

CO 

OlA) 

0 

0 

CO 

0  ) 

CO 

023) 

0 

302 

CO 

006) 

CO 

018) 

0 

0 

CO 

0  ) 

CO 

030) 

0 

0 

CO 

0  ) 

CO 

OlA) 

0 

0 

CO 

0  ) 

CO 

027) 

0 

155 

CO 

006) 

CO 

050) 

0 

275 

CO 

007) 

CO 

061) 

0 

277 

CO 

006) 

"579- 


625 


Table  E.IO 

1986  IKT  Parameters,  Mathematics,  Grade  7/Age  13 
Geometry  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

B 

SE 

c 

SE 

N2SA801 

Ml 

16 

0. 

789 

(0 . 

0A3  ) 

-1. 

857 

(0 . 

10  6) 

0  . 

239 

(0  . 

031) 

N2SA602 

Ml 

A6 

1. 

869 

( 0 . 

061 ) 

0. 

621 

(0 . 

05A  ) 

0  . 

228 

( 0 . 

009 ) 

N270301 

M2 

20 

0. 

771 

(0 . 

039 ) 

-1. 

0A8 

(0  . 

059 ) 

0  . 

205 

( 0 . 

020  ) 

N270302 

M2 

21 

2. 

133 

(0 . 

0A7  ) 

0. 

869 

(0 . 

050  ) 

0 

055 

( 0 . 

005 ) 

N253701 

M2 

22 

0. 

5A3 

(0 . 

056 ) 

-0. 

A57 

(0 . 

058 ) 

0 

258 

(0  . 

019 ) 

N2SA001 

M3 

28 

0 

722 

(0. 

0A5) 

-0 

A65 

(0. 

039 ) 

0 

189 

(0 . 

017 ) 

N269901 

M3 

29 

0 

761 

(0 . 

099 ) 

-0 

102 

(0 . 

OA  5) 

0 

251 

( 0 . 

019 ) 

N26A601 

MA 

3A 

1 

305 

(0 . 

050 ) 

1 

1A3 

(0 . 

072 ) 

0 

357 

(0  . 

008 ) 

N25A501 

M5 

35 

0 

8A2 

(0 

053 ) 

-0 

090 

(0 . 

031) 

0 

2A1 

(0 

015) 

N253202 

M5 

37 

1 

27A 

\  0 

0A9 ) 

0 

997 

( 0 

063 ) 

0 

188 

V  U 

010) 

N253801 

M5 

42 

1 

311 

(0 

061) 

2 

918 

(0 

190) 

0 

080 

(0 

005) 

N25A301 

M7 

35 

1 

228 

(0 

039) 

0 

872 

(0 

0A8) 

0 

256 

(0 

007  ) 

N213101 

M7 

36 

1 

002 

(0 

129) 

1 

lAA 

(0 

16A) 

0 

257 

(0 

009) 

N21A901 

M7 

Al 

0 

.39A 

(0 

021) 

1 

.262 

(0 

070) 

0 

165 

(0 

009) 

N226201 

M7 

A7 

1 

.969 

(0 

051) 

1 

.555 

(0 

08A) 

0 

.3A8 

(0 

006) 

N21A701 

MS 

22 

0 

.573 

(0 

027  ) 

0 

.259 

(0 

025) 

0 

.279 

(0 

010) 

N23A901 

M8 

32 

1 

.283 

(0 

OA^) 

0 

.A69 

<0 

033  ) 

0 

.176 

(0 

.007  ) 

N251701 

M8 

AO 

1 

.0A6 

(0 

0A6) 

0 

.682 

(0 

0A7) 

0 

.170 

(0 

.009) 

N213601 

MS 

Al 

1 

.763 

(0 

.  0A8  ) 

0 

.771 

(0 

.  0A6) 

0 

.156 

(0 

.007  ) 

N212&01 

M9 

32 

1 

.650 

(0 

.052) 

-0 

.661 

(0 

.036) 

0 

.236 

(0 

.OlA) 

N212902 

M9 

33 

1 

.5A5 

(0 

.056) 

0 

.103 

(0 

.025) 

0 

.  121 

(0 

.008) 

N212903 

M9 

3A 

1 

.  A78 

(0 

.055) 

0 

.323 

(0 

.03A) 

0 

.281 

(0 

.007  ) 

N226A01 

M9 

35 

0 

.698 

(0 

.030) 

0 

.27A 

(0 

.026) 

0 

.287 

(0 

.009) 

N215001 

M9 

39 

0 

.6A3 

(0 

.025) 

0 

.036 

(0 

.020) 

0 

.189 

(0 

.010) 

Table  £.11 

1986  IRT  Parameters,  Mathematics,  Grade  ll/A&e  17 
Measurement  Subscale 


FIELD 

BLOCK 

ITEM 

N26A301 

Ml 

A7 

N251101 

Ml 

A9 

N266801 

M2 

16 

N269001 

M2 

22 

N261801 

M2 

25 

N268901 

M2 

A7 

N268801 

M2 

A8 

N266S01 

M3 

31 

N285201 

MA 

29 

N266701 

MA 

32 

N251201 

M5 

26 

N28AA01 

M5 

27 

N252201 

M5 

30 

N265903 

M5 

31 

N251801 

M5 

32 

N266001 

M5 

38 

N252601 

M5 

AO 

N267901 

M5 

Al 

N266101 

M6 

2A 

N265901 

M6 

39 

N269201 

M6 

Al 

N265902 

M6 

A2 

N252701 

M6 

AA 

N218501 

M7 

20 

N216A01 

M7 

28 

N216301 

M7 

30 

N216101 

M7 

33 

N217101 

M7 

3A 

N215701 

M8 

18 

N215601 

M8 

26 

N232101 

M8 

Al 

N231801 

M8 

A3 

N216201 

MS 

A3 

N231501 

MIO 

13 

N232901 

MIO 

15 

N267801 

MIO 

20 

N230701 

MIO 

28 

N21900X 

MIO 

30 

N218801 

Mil 

22 

ERIC 


A 

SE 

B 

SE 

c 

SE 

1 

A88 

(0 

.  022) 

1 

506 

(0 

.035) 

0 

0 

(0 

.0  ) 

1 

57A 

(0 

.  023 ) 

1 

662 

(0 

.0A2) 

0 

0 

(0 

0  ) 

0 

800 

(0 

.  030) 

-0 

188 

(0 

.023) 

0 

205 

(0 

016) 

1 

A68 

(0 

.  033) 

0 

521 

(0 

.026) 

0 

115 

(0 

010) 

0 

915 

(0 

.  03A ) 

-0 

2A5 

(0 

.02A) 

0 

195 

(0 

017) 

2 

801 

(0 

.  0A6) 

1 

A13 

(0 

.070) 

0 

209 

(0 

012) 

2 

AA3 

(0 

.  038) 

1 

83A 

(0 

.071) 

0 

091 

(0 

008) 

1 

165 

(0 

.  032) 

0 

670 

(0 

.037) 

0 

226 

(0 

016) 

1 

291 

(0 

.  032) 

0 

205 

(0 

.018) 

0 

0 

(0 

0  ) 

0 

820 

(0 

.  023) 

1 

A21 

(0 

.0A7) 

0 

210 

(0 

009) 

0 

781 

( 0 

.  021) 

0 

993 

(0 

.035) 

0 

162 

ro 

010) 

1 

028 

( 0 

.  023 ) 

0 

030 

(0 

.015) 

0 

0 

(0 

0  ) 

1 

331 

( 0 

.  023 ) 

0 

802 

(0 

.  02A  ) 

0 

0 

(0 

0  ) 

0 

886 

( 0 

.  025) 

1 

050 

(0 

.039) 

0 

257 

(0 

009) 

1 

llA 

( 0 

.  022) 

0 

819 

(0 

.  02A  ) 

0 

0 

(0 

0  ) 

1 

A75 

(0 

.  020) 

1 

713 

(0 

.036) 

0 

0 

(0 

0  ) 

2 

112 

(0 

.031) 

1 

AA5 

(0 

.0A9) 

0 

210 

(0 

006) 

1 

078 

( 0 

.  020) 

2 

105 

(0 

.0A9) 

0 

0 

(0 

0  ) 

1 

275 

( 0 

.035) 

0 

576 

(0 

.031) 

0 

29A 

(0 

008) 

1 

6A3 

( 0 

.  033 ) 

1 

088 

(0 

.  0A2) 

0 

297 

(0 

008) 

1 

693 

( 0 

.  023 ) 

1 

186 

(0 

.029) 

0 

0 

(0 

0  ) 

2 

227 

( 0 

.  OA  1) 

1 

A16 

(0 

.06A) 

0 

399 

(0 

007) 

1 

A37 

( 0 

.  021 ) 

1 

361 

(0 

.  032) 

0 

0 

(0 

0  ) 

1 

811 

( 0 

.  028 ) 

1 

110 

(0 

.  037 ) 

0 

3AA 

(0 

007) 

2 

Oil 

( 0 

030 ) 

0 

778 

n  o  Q  ^ 

0 

156 

( 0 

008 ) 

1 

538 

(0 

.023) 

1 

A05 

(0 

039) 

0 

28A 

(0 

006) 

1 

963 

(0 

.026) 

1 

069 

(0 

03A) 

0 

133 

(0 

009) 

2 

307 

(0 

.026) 

1 

AA2 

(0 

OAl) 

0 

175 

(0 

005) 

1 

AOO 

(0 

.029) 

0 

566 

(0 

02A) 

0 

220 

(0 

008) 

1 

628 

(0 

.028) 

0 

896 

(0 

031) 

0 

270 

(0 

007) 

1 

815 

(0 

.030) 

1 

923 

(0 

.060) 

0 

173 

(0 

007) 

1 

296 

(0 

.031) 

1 

085 

(0 

.0A2) 

0 

All 

(0 

007  ) 

1 

176 

(0 

.023) 

1 

307 

(0 

039) 

0 

182 

(0 

010) 

0 

699 

(0 

.020) 

0 

355 

(0 

019) 

0 

233 

(0 

010) 

1 

795 

(0 

.026) 

0 

760 

(0 

.02A) 

0 

150 

(0 

006) 

1 

255 

(0 

.022) 

1 

233 

(0 

.035) 

0 

207 

(0 

009) 

2 

188 

(0 

.038) 

1 

912 

(0 

.07A) 

0 

326 

(0 

007  ) 

1 

BOA 

(0 

.038) 

0 

69A 

(0 

.038) 

0 

211 

(0 

013) 

1 

AC8 

(0 

.025) 

0 

966 

(0 

032) 

0 

158 

(0 

010) 

-581- 

627 


Table  E.12 

1986  IRT  Parameters,  Mathematics,  Grade  11/Age  17 
Numbers  and  Operations — High  Level  Subscale 


FIELD      BLOCK  ITEM 


N286302 

Ml 

22 

N2S8802 

Ml 

26 

N2S9901 

Ml 

28 

N260901 

Ml 

35 

N261S01 

M2 

2A 

N261201 

M2 

26 

N261601 

M2 

27 

N261301 

M2 

28 

N281A01 

M2 

29 

N2S9001 

M2 

31 

N286301 

M2 

33 

N258801 

M2 

38 

N260801 

M2 

A3 

N255301 

M2 

A6 

N271301 

M3 

32 

N263901 

MA 

30 

N255302 

MA 

37 

N258201 

MA 

39 

N260902 

MA 

AO 

N285*01 

MA 

Al 

N257i>  U 

M5 

23 

N279301 

M5 

3A 

N259501 

M5 

36 

N286201 

M6 

23 

N282201 

M6 

27 

N256501 

M6 

35 

N23A201 

M7 

21 

N201A01 

M7 

23 

N201A02 

M7 

2A 

N201101 

M7 

25 

N202501 

M7 

27 

N204101 

M8 

16 

N208101 

M8 

17 

N206601 

M8 

20 

N207101 

M8 

2A 

N206501 

M8 

25 

N201701 

M8 

30 

H20  0101 

M8 

31 

N200901 

M8 

3A 

N206801 

M9 

38 

N262001 

M9 

A2 

N203801 

M9 

A6 

N204501 

M9 

A8 

N234101 

M9 

53 

N205501 

MIO 

11 

N205201 

MIO 

12 

N208301 

MIO 

19 

N205901 

MIO 

22 

N206701 

MIO 

26 

N205801 

MIO 

31 

N227101 

MIO 

35 

N205101 

Mil 

lA 

N207801 

Mil 

15 

N204A01 

Mil 

21 

N202701 

Mil 

AO 

ERIC 


A 

SE 

B 

SE 

c 

SE 

1. 

503 

(0 . 

03A  ) 

0. 

726 

(0. 

OAO ) 

0. 

289 

(0 . 

017 ) 

2. 

279 

(0  . 

03A  ) 

0  . 

808 

(0 . 

033 ) 

0  . 

215 

(0 . 

008 ) 

1. 

7A2 

(0  . 

026 ) 

0  . 

801 

(0 . 

027 ) 

0  . 

199 

(0 . 

007 ) 

3. 

161 

(0 . 

037  ) 

1. 

170 

(0 . 

0  A  6 ) 

0. 

182 

(0 . 

007 ) 

1. 

081 

(0 . 

038 ) 

-0. 

506 

(0 . 

028) 

0. 

130 

(0 . 

019 ) 

0. 

935 

(0 . 

02A ) 

0. 

21A 

(0 . 

020 ) 

0. 

150 

(0  . 

0 13 ) 

0. 

808 

(0  . 

020 ) 

1. 

393 

(0 . 

OAl) 

0. 

222 

( 0 . 

008 ) 

0. 

9A0 

(0  . 

022) 

0. 

272 

(0 . 

019) 

0. 

109 

(0  . 

0 12 ) 

1. 

110 

(0  . 

020 ) 

0. 

890 

(0 . 

025 ) 

0. 

107 

(0  . 

009) 

1. 

608 

(0  . 

023 ) 

0. 

871 

(0  . 

027 ) 

0. 

0 

(0 . 

0  ) 

2. 

002 

(0  . 

032 ) 

0  . 

758 

(0  . 

031 ) 

0. 

197 

(0 . 

009 ) 

2. 

lOA 

(0  . 

032 ) 

1. 

170 

(0 . 

0  A2 ) 

0. 

326 

(0  . 

007 ) 

1. 

731 

(0  . 

021 ) 

1 . 

358 

(0  . 

032) 

0. 

0 

(0  . 

0  ) 

2. 

05A 

(0  . 

050) 

2. 

258 

(0  . 

113 ) 

0. 

231 

(0  . 

008 ) 

2. 

1A7 

(0  . 

031 ) 

1. 

249 

(0  . 

0  A  A  ) 

0. 

303 

(0  . 

007 ) 

1. 

58A 

(0. 

026) 

0. 

726 

(0  . 

026) 

0. 

153 

(0  . 

008 ) 

1. 

809 

(0. 

026) 

1. 

AA3 

(0. 

0A2) 

0. 

211 

(0  . 

007 ) 

1. 

065 

(0. 

021) 

0  . 

618 

(0. 

022) 

0. 

0 

(0  . 

0  ) 

2. 

739 

(0  . 

OAl ) 

1. 

288 

(0  . 

057 ) 

0. 

289 

(0  . 

009 ) 

1. 

378 

(0. 

021 ) 

1. 

808 

(0  . 

039) 

0. 

0 

(0  . 

0  ) 

2. 

217 

(0  . 

05A  ) 

0 

512 

(0  . 

040) 

0. 

3A0 

(0  . 

012 ) 

1 

357 

(0  . 

012) 

1 

573 

(0. 

021) 

0 

0 

(0  . 

0  ) 

1 

292 

(0 

019 ) 

0 

972 

(0 

02A  ) 

0 

0 

(0 

0  ) 

0 

98A 

(0 

03A  ) 

-0 

095 

(0 

022) 

0 

203 

(0 

015 ) 

1 

53A 

(0 

026) 

1 

OAA 

(0 

03A ) 

0 

26A 

(0 

008 ) 

2 

150 

(0 

038) 

0 

890 

(0 

OAl) 

0 

318 

(0 

009 ) 

1 

7  92 

(0 

022 ) 

0 

797 

(0 

022) 

0 

175 

(0 

006 ) 

1 

010 

(0 

015 ) 

0 

985 

(0 

021) 

0 

071 

( 0 

006 ) 

0 

993 

(0 

016) 

0 

.601 

(0 

017 ) 

0 

079 

(0 

008) 

1 

A6A 

(0 

.  021 ) 

0 

.701 

( 0 

.  022 ) 

0 

158 

r  n 

008) 

1 

.296 

1 

.036 

t  n 

n9  A  ^ 

.  UZ*4  } 

0 

.  197 

006) 

1 

.850 

-0 

.012 

r  n 

n  1  Q  ^ 

0 

.  17A 

n  1  n  ^ 
.  U  X  u  ^ 

1 

.390 

( 0 

.  025  ) 

0 

.  A2A 

( 0 

.  020  ) 

0 

.  163 

(0 

n  n  Q  ^ 

0 

.70A 

( 0 

.015) 

0 

.012 

(0 

.011) 

0 

.075 

( 0 

.007  ) 

1 

.  822 

(0 

.  025  ) 

1 

.0A2 

( 0 

.  031 ) 

0 

.3A8 

( 0 

.006) 

1 

.071 

(0 

.  023 ) 

0 

.2A6 

(0 

.  017 ) 

0 

.  193 

(0 

.  0 10  ) 

1 

.  A32 

(0 

.023  ) 

0 

.897 

(0 

.  028 ) 

0 

.306 

(0 

.  007  ) 

1 

.563 

( 0 

.  039 ) 

0 

.  155 

(0 

.  027  ) 

0 

.  181 

(0 

.018) 

1 

.829 

(0 

.  021 ) 

1 

.  071 

(0 

.  027 ) 

0 

.  192 

(0 

.  006 ) 

2 

.  1A3 

(0 

.  031 ) 

0 

.596 

(0 

.  025) 

0 

.  192 

(0 

.  007  ) 

2 

.200 

( 0 

.  032 ) 

0 

.872 

( 0 

.036) 

0 

.216 

0 

.879 

(0 

.022) 

1 

.68A 

(0 

.051) 

0 

.339 

(0 

.006) 

1 

.  892 

(0 

.026) 

1 

.03A 

(0 

.03A) 

0 

.  194 

(0 

.011) 

0 

.931 

(0 

.022) 

0 

.211 

(0 

.016) 

0 

.  152 

(0 

.007) 

0 

.  795 

(0 

.019) 

2 

.0A8 

(0 

.057) 

0 

.227 

(0 

.007  ) 

1 

.  A53 

(0 

.023) 

0 

.531 

(0 

.019) 

0 

.  188 

(0 

.007) 

2 

.072 

(0 

.027) 

1 

.  87A 

(0 

.052) 

0 

.2A1 

(0 

.006) 

1 

.529 

(0 

.020) 

1 

.326 

(0 

.032) 

0 

.267 

(0 

.006) 

1 

.363 

(0 

.020  ) 

1 

.0A3 

(0 

.027) 

0 

.225 

(0 

.007  ) 

2 

.727 

(0 

.031) 

1 

.668 

(0 

.053) 

0 

.  189 

(0 

.007) 

1 

.266 

(0 

.029) 

1 

.759 

(0 

.059) 

0 

.2A1 

(0 

.009) 

2 

.690 

(0 

.033) 

0 

.922 

(0 

.035) 

0 

.376 

(0 

.006) 

1 

.735 

(0 

.021) 

0 

.779 

(0 

.021) 

0 

.213 

(0 

.OOA  ) 

2 

.399 

(0 

.02A) 

1 

.063 

(0 

.029) 

0 

.  105 

(0 

.007  ) 

2 

.959 

(0 

.03A) 

1 

.771 

(0 

.059) 

0 

.076 

(0 

.005) 

-582- 
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Table  E.13 

1986  IRT  Parameters,  Mathematics,  Grade  11/Age  17 
Numbers  and  Operations—Knowledge  and  Skills  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N260601 

Ml 

16 

1 . 

283 

(0 . 

019) 

N25880A 

Ml 

18 

1 . 

268 

( 0 . 

030 ) 

N278501 

Ml 

23 

3 . 

117 

(0 

031) 

N278502 

Ml 

2A 

2. 

6A7 

(0 

02A  ) 

N278503 

Ml 

25 

2 . 

692 

(0 

032) 

N2d7 101 

Ml 

29 

2. 

382 

(0 

035) 

N286S02 

Ml 

3A 

2 . 

908 

(0 

036) 

N258803 

Ml 

37 

1. 

7A7 

(0 

022) 

N278905 

Ml 

A  A 

2 . 

237 

(0 

033) 

N287301 

Ml 

A5 

1 . 

195 

(0 

017) 

K287  302 

Ml 

A6 

1 . 

572 

(0 

019) 

Nzou 10 1 

M2 

20 

2 

A38 

(0 

035) 

NzoOhO 1 

M2 

30 

1 . 

00  A 

(0 

019) 

n287 102 

M2 

32 

2 

600 

(0 

038) 

NzobDO 1 

M2 

3  A 

2 

388 

(0 

OAl) 

ur  O  C  1  A  A  1 

NzblOO 1 

M2 

AO 

1 

326 

(0 

026) 

N256301 

MA 

19 

1 

126 

(0 

019) 

N280601 

M4 

23 

1 

759 

( 0 

018) 

n2o0602 

MA 

2A 

1 

935 

(0 

020) 

N280603 

MA 

25 

1 

682 

( 0 

025) 

N28060A 

MA 

26 

1 

913 

( 0 

017) 

N280605 

MA 

27 

2 

018 

( 0 

022 ) 

N280606 

MA 

28 

1 

61A 

(0 

016 ) 

N27  1<»01 

MA 

33 

1 

815 

( 0 

029 ) 

N278301 

MA 

35 

1 

A  A5 

(0 

019 ) 

N278302 

MA 

36 

1 

7A3 

( 0 

017) 

N28A101 

M5 

18 

0 

670 

(0 

035) 

N28A 102 

M5 

19 

0 

596 

( 0 

021 ) 

N28A  501 

M5 

20 

0 

798 

(0 

015 ) 

N28A502 

M5 

21 

1 

159 

(0 

.026) 

n2oA  503 

H5 

22 

1 

59A 

(0 

.022) 

MO  T  'J  Q  n  o 

M5 

25 

1 

991 

(0 

.028) 

M  9  Q  ^  n  n  1 

O  Q 
ZO 

1 

331 

(0 

.021) 

M9  7  A  ft  no 

no 

O  Q 
Zo 

o 
z 

350 

(0 

.031) 

MOKn  7  n 1 
n  ZD  u / u  X 

clD 

1 
1 

R  Q 
DsO 

(0 

.020) 

iio  7  7  qn  1 

M6 

1  A 

i.  S 

A 
U 

fl  Q  Q 

(0 

.OlA) 

MO  7  7  ono 

aZ.  /  /  ^ UZ 

MR 
[ID 

1 

0 

893 

(0 

.OlA) 

MO  7  7  Q  m 

riD 

1  R 
ID 

1 

017 

(0 

.016) 

MO  7  R  Q  n 1 

MR 
PID 

1  7 

0 

503 

(0 

.019) 

MO  7  C  Q  n  o 

Nz / boUz 

M6 

18 

0 

.  769 

(0 

.017) 

MO  7  K  p  n 

nz / Douo 

MR 

Mb 

1  9 

0 

.  8  A 1 

(0 

.018) 

IJO  7  7  C  A  1 

Nz /  / bu 1 

M6 

20 

1 

.  288 

(0 

.021) 

U  O  T  T  C  A  O 

Nz /  / bOz 

Mb 

21 

1 

.300 

(0 

.025) 

MO  7  7  fifl'^ 

M6 

22 

.  23 A 

(0 

.023) 

N27<.801 

M6 

25 

1 

.291 

(0 

.031) 

N286601 

M6 

28 

1 

.  538 

(0 

.023) 

N286602 

M6 

29 

1 

.311 

(0 

.020) 

N286603 

M6 

30 

1 

.935 

(0 

.021) 

N283101 

M6 

AO 

2 

.071 

(0 

.025) 

N230201 

M8 

35 

0 

.SIA 

(0 

.017) 

N201001 

M8 

A2 

3 

.8A9 

(0 

.0A6) 

N282202 

M9 

3A 

1 

.083 

(0 

.02A) 

N28A001 

Mil 

12 

1 

.  162 

(0 

.022) 

N28A002 

Mil 

13 

1 

.  738 

(0 

.028) 

N225901 

Mil 

17 

1 

.676 

(0 

.021) 

N200201 

Mil 

26 

1 

.  7A9 

(0 

.022) 

B  SE  C  SE 


0  . 

AA2 

(0. 

OlA) 

0 

0 

(0. 

0 

0  . 

200 

(0. 

020) 

0. 

376 

(0. 

009 

0  . 

A29 

(0. 

019) 

0. 

0 

(0. 

0 

0 . 

525 

(0. 

017) 

0. 

0 

(0. 

0 

0 . 

337 

(0. 

018) 

0. 

0 

(0. 

0 

0 . 

805 

(0. 

035) 

0. 

296 

(0. 

oil 

0 . 

83A 

(0. 

035) 

0 

192 

(0. 

oil 

1 . 

030 

(0. 

026) 

0. 

167 

(0. 

007 

1 . 

A67 

(0 

050) 

0. 

275 

(0 

007 

0 

803 

(0 

018) 

0. 

0 

(0 

0 

1 . 

7A1 

(0 

030) 

0 

0 

(0. 

0 

0 

630 

(0 

028) 

0 

237 

(0. 

009 

0 

235 

(0 

OlA  ) 

0 

0 

(0 

0 

0 

658 

(0 

033) 

0 

162 

(0 

013 

0 

530 

(0 

033) 

0 

172 

(0 

016 

0 

555 

(0 

02A) 

0 

236 

(0 

oil 

0 

237 

(0 

OlA) 

0 

0 

(0 

0 

0 

67A 

(0 

016) 

0 

0 

(0 

0 

0 

66A 

(0 

017) 

0 

0 

(0 

0 

0 

2AA 

(0 

016) 

0 

0 

(0 

0 

0 

912 

(0 

017) 

0 

0 

(0 

0 

0 

537 

(0 

017) 

0 

0 

(0 

0 

0 

973 

(0 

017) 

0 

0 

(0 

0 

1 

218 

(0 

OAO) 

0 

A05 

(0 

008 

0 

603 

(0 

017) 

0 

0 

(0 

0 

1 

106 

(0 

020) 

0 

0 

(0 

0 

-2 

177 

(0 

119) 

0 

0 

(0 

0 

-0 

776 

(0 

032) 

0 

0 

(0 

0 

0 

156 

(0 

012) 

0 

0 

(0 

0 

-0 

361 

(0 

018) 

0 

0 

(0 

0 

0 

301 

( 0 

015) 

0 

0 

(0 

0 

0 

622 

( 0 

02A  ) 

0 

1A5 

(0 

009 

0 

528 

( 0 

018 ) 

0 

0 

(0 

0 

0 

637 

( 0 

026 ) 

0 

161 

(0 

009 

0 

869 

( 0 

.  020 ) 

0 

0 

(0 

0 

-1 

871 

( 0 

03A  ) 

0 

0 

(0 

0 

"•1 

839 

( 0 

033 ) 

0 

0 

(0 

0 

- 1 

0A2 

( 0 

.  021 ) 

0 

0 

(0 

0 

-3 

.780 

(0 

.  1A3 ) 

0 

.  0 

(0 

0 

-2 

.066 

(0 

.  0A9) 

0 

.  0 

(0 

0 

~0 

.  6A2 

(0 

.019) 

0 

.  0 

(0 

0 

~  1 

.101 

( 0 

.  025 ) 

0 

.  0 

(0 

.  0 

-0 

.  AA5 

(0 

.017) 

0 

.0 

(0 

.0 

"0 

.  55A 

(0 

.018) 

0 

.0 

(0 

.0 

0 

.  A78 

(0 

.027) 

0 

.  A19 

(0 

.010 

0 

A38 

(0 

.017) 

0 

.0 

(0 

.0 

0 

.538 

(0 

.017) 

0 

.0 

(0 

.0 

0 

.787 

(0 

.019) 

0 

.0 

(0 

.0 

1 

.07A 

(0 

.030) 

0 

.  183 

(0 

.007 

1 

.165 

(0 

.027) 

0 

.155 

(0 

.006 

1 

.335 

(0 

.061) 

0 

.277 

(0 

.006 

0 

.333 

(0 

.020) 

0 

.310 

(0 

.011 

-0 

.8A8 

(0 

.02A) 

0 

.0 

(0 

.0 

0 

.125 

(0 

.018) 

0 

.0 

(0 

.0 

0 

.909 

(0 

.023) 

0 

.302 

(0 

.006 

1 

.263 

(0 

.030) 

0 

.20A 

(0 

.007 
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Table  E.IA 

1986  IRT  Parameters,  Mathematics,  Grade  11/Age  17 
Geometry  Subocale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N2SA602 

Ml 

27 

1 . 

869 

(0. 

061) 

N270301 

Ml 

30 

0 . 

771 

(0. 

039) 

N270302 

Ml 

31 

2 . 

133 

(0. 

0A7) 

N25A301 

Ml 

33 

1 . 

228 

(0. 

039) 

N2S3901 

Ml 

39 

1 . 

707 

(0. 

058) 

N253902 

Ml 

AO 

0  . 

505 

(0. 

032) 

N253903 

Ml 

Al 

2  . 

OAS 

(0. 

051) 

N25390A 

Ml 

42 

1 . 

210 

(0. 

038) 

N2SA601 

M2 

15 

0  . 

789 

(0. 

0A3) 

N25A001 

M2 

21 

0 

722 

(0. 

0A5) 

N25170 1 

M2 

Al 

1 

0A6 

(0. 

0A6) 

N26A601 

MA 

3A 

1 

305 

(0. 

050) 

N25A501 

M5 

35 

0 

8A2 

(0 

053) 

W253202 

H5 

37 

1 

27A 

(0 

0A9) 

N253801 

M5 

A2 

1 

311 

(0 

061) 

N219301 

M7 

19 

1 

603 

(0 

03A ) 

N212901 

M8 

21 

1 

650 

(0 

052) 

N212902 

M8 

22 

1 

5A5 

(0 

056) 

N212903 

M8 

23 

1 

A78 

(0 

055) 

N213001 

M8 

27 

1 

5A2 

(0 

0A8) 

N213201 

M8 

29 

0 

8A9 

(0 

030) 

N23A901 

M8 

37 

1 

.283 

(0 

0A2) 

N226A01 

M9 

35 

0 

.698 

(0 

030) 

N215001 

M9 

37 

0 

.6A3 

(0 

.025) 

N21A701 

M9 

39 

0 

.573 

(0 

.027) 

N213601 

M9 

AO 

1 

.763 

(0 

.OAP) 

N21A901 

M9 

A5 

0 

.39A 

(0 

.021) 

N213A01 

M9 

A9 

1 

.  1A2 

(0 

.  OAO) 

N213501 

M9 

50 

0 

.693 

(0 

.033) 

N212701 

MIO 

21 

0 

.800 

(0 

.031) 

N226201 

MIO 

25 

1 

.969 

(0 

.051) 

N232001 

MIO 

27 

1 

.217 

(0 

.038) 

N215101 

MIO 

29 

1 

.087 

(0 

.032) 

N213701 

Mil 

28 

1 

.518 

(0 

.038) 

N21A501 

Mil 

30 

0 

.822 

(0 

.036) 

N21A801 

Mil 

38 

1 

.03A 

(0 

.0A2) 

N230801 

Mil 

Al 

1 

.  A02 

(0 

.086) 

B  SE  C  SE 


0 . 

621 

(0. 

05A) 

C. 

228 

(0. 

009) 

-1. 

048 

(0. 

059) 

0 . 

205 

(0. 

020) 

0 . 

869 

(0. 

050) 

0 . 

055 

(0  . 

005) 

0 . 

872 

(0  . 

0A8) 

0 . 

256 

(0 . 

007) 

0 . 

6A3 

(0. 

060) 

0 . 

216 

(0. 

015) 

1 . 

205 

(0. 

088) 

0  . 

3A7 

(0 . 

016) 

1 . 

450 

(0. 

079) 

0 . 

318 

(0 . 

012) 

2A8 

(0. 

066) 

0 . 

299 

(0. 

013) 

-1. 

857 

(0. 

106) 

0  . 

239 

(0. 

031) 

-0 . 

465 

(0. 

039) 

0  . 

189 

(0 . 

017) 

0 . 

682 

(0. 

0A7) 

0  . 

170 

(0 . 

009) 

1. 

1A3 

(0. 

072) 

0  . 

357 

(0  . 

008) 

-0. 

090 

(0. 

031) 

0 

2A1 

(0. 

015) 

0 

997 

(0 

063) 

0 

188 

(0 

010) 

2 

918 

(0 

190) 

0 

080 

(0 

005) 

1 

32A 

(0 

050) 

0 

077 

(0 

007) 

-0 

661 

(0 

036) 

0 

236 

(0 

014) 

0 

103 

(0 

025) 

0 

121 

(0 

008) 

0 

323 

(0 

03A) 

0 

281 

(0 

007) 

0 

719 

(0 

050) 

0 

251 

(0 

010) 

0 

735 

(0 

OAl) 

0 

219 

(0 

Oil) 

0 

A89 

(0 

033) 

0 

176 

(0 

007) 

0 

.27A 

(0 

026) 

0 

287 

(0 

009) 

0 

.036 

(0 

020) 

0 

.  189 

(0 

.010) 

0 

.259 

(0 

.025) 

.279 

(0 

.010) 

0 

.771 

(0 

.  0A6) 

0 

.  156 

(0 

.007) 

1 

.262 

(0 

.070) 

0 

.165 

(0 

.009) 

0 

.909 

(0 

.055) 

0 

.228 

(0 

.012) 

1 

.15A 

(0 

.069) 

0 

.271 

(0 

.OlA) 

0 

.872 

(0 

.0A7) 

0 

.237 

(0 

.012) 

1 

.555 

(0 

.08A  ) 

0 

.3A8 

(0 

.006) 

1 

.806 

(0 

.08A) 

0 

.307 

(0 

.009) 

1 

.881 

(0 

.07A) 

0 

.  lAO 

(0 

.008) 

1 

.397 

(0 

.061) 

0 

.  188 

(0 

.008) 

2 

.899 

(0 

.  1A3) 

0 

.  118 

(0 

.006) 

1 

.780 

(0 

.095) 

0 

.2A2 

(0 

.010) 

2 

.736 

(0 

.25A) 

0 

.266 

(0 

.009) 
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Table  E.15 

1986  IRT  Parameters,  Mathematics,  Grade  11/Age  17 
Relations  and  Functions  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N2S6101 

Ml 

15 

1 

101 

n  A  A  ^ 

N255701 

Ml 

32 

1 

3A5 

1  n  7  \ 

N282801 

Ml 

A8 

1 

733 

n  7Q  \ 

\J  19  } 

N26A701 

M2 

39 

1 

202 

1  n*!  \ 
1  Uo  ; 

N255601 

M2 

A5 

1 

0A5 

( 0 

0  72 ) 

N255501 

M3 

33 

0 

923 

( 0 

0  55 ) 

N256001 

M3 

3A 

0 

926 

( 0 

052 ) 

N257101 

M3 

35 

0 

928 

071) 

N255101 

MA 

38 

1 

170 

( 0 

082 ) 

N282701 

M5 

24 

0 

6AA 

( 0 

057 ) 

N270701 

M6 

37 

2 

282 

1  o  A  ; 

N270702 

M6 

38 

1 

092 

(  0 

057 ) 

N255A01 

M6 

A3 

1 

835 

(  0 

0  89) 

N285901 

M6 

A6 

1 

527 

(  0 

069 ) 

N209A01 

M8 

32 

1 

057 

t  n 

N210901 

M8 

36 

1 

237 

U  Do  ^ 

N208601 

M8 

38 

0 

8A2 

038  ) 

N210101 

M8 

A6 

1 

683 

n Rn  \ 

N208501 

M8 

A7 

1 

A35 

(  0 

0  62  ) 

N209801 

M9 

AA 

0 

73A 

n  ^n  ^ 

N211901 

M9 

51 

0 

92A 

U  oZ  } 

N210601 

MIO 

lA 

1 

256 

( 0 

057  ) 

N210701 

MIO 

16 

1 

302 

n  R  n  ^ 

N209601 

MIO 

17 

0 

922 

(0 

038) 

N209S01 

MIO 

18 

1 

199 

(0 

059) 

N233401 

MIO 

23 

0 

865 

(0 

0A7) 

N233A02 

MIO 

2A 

0 

913 

(0 

.0A9) 

N211801 

MIO 

32 

1 

llA 

(0 

106) 

N25590] 

MIO 

33 

0 

9A1 

(0 

066) 

N255902 

MIO 

3A 

1 

66A 

(0 

077) 

N20d801 

Mil 

16 

0 

9A0 

(0 

.036) 

N226001 

Mil 

18 

1 

877 

(0 

.082) 

N210A01 

Mil 

20 

2 

057 

(0 

.102) 

N209301 

Mil 

23 

1 

760 

(0 

.065) 

N209901 

Mil 

2A 

1 

520 

(0 

.057) 

N210301 

Mil 

25 

1 

A93 

(0 

.060) 

N208901 

Mil 

3A 

0 

535 

(0 

.038) 

N229901 

Mil 

37 

1 

671 

(0 

.055) 

B  SE  C  SE 


-1 

338 

(0 

066) 

0 

0 

( -J 

0  ) 

-0 

371 

(0 

053) 

0 

216 

(0 

021) 

1 

A6A 

(0 

121) 

0 

209 

(0 

oil) 

0 

100 

(0 

048) 

0 

199 

(0 

018) 

1 

502 

(0 

136) 

0 

285 

(0 

013) 

0 

933 

(0 

076) 

0 

283 

(0 

013) 

0 

298 

(0 

031) 

0 

0 

(0 

0  ) 

2 

18A 

(0 

201) 

0 

310 

(0 

oil) 

0 

5A6 

(0 

066) 

0 

221 

(0 

015) 

-0 

8A9 

(0 

082) 

0 

237 

(0 

02A) 

-0 

029 

(0 

061) 

0 

113 

(0 

OlA  ) 

0 

602 

(0 

055) 

0 

176 

(0 

015) 

1 

93A 

(0 

177) 

0 

23A 

(0 

010) 

1 

051 

(0 

085) 

0 

179 

(0 

013) 

-0 

02A 

(0 

032) 

0 

198 

(0 

OlA  ) 

0 

663 

(0 

051) 

0 

21A 

(0 

009) 

0 

879 

(0 

051) 

0 

12A 

(0 

009) 

1 

A29 

(0 

092) 

0 

172 

(0 

.008) 

1 

038 

(0 

077  ) 

0 

185 

(0 

009) 

0 

335 

(0 

039) 

0 

213 

(0 

013) 

1 

lAl 

(0 

097) 

0 

330 

(0 

Oil) 

-0 

A45 

(0 

037) 

0 

162 

(0 

016) 

-0 

34A 

vO 

036) 

0 

176 

(0 

015) 

,1 

224 

(0 

063) 

0 

156 

(0 

008) 

0 

053 

(0 

030) 

0 

156 

(0 

012) 

-0 

219 

(0 

031) 

0 

231 

(0 

015) 

0 

095 

(0 

031) 

0 

233 

(0 

013) 

3 

159 

(0 

379) 

0 

168 

(0 

007  ) 

2 

350 

(0 

200) 

0 

217 

(0 

009) 

1 

461 

(0 

121) 

0 

228 

(0 

010) 

0 

131 

(0 

027) 

0 

123 

(0 

Oil) 

-0 

273 

(0 

037) 

0 

096 

(0 

.013) 

-0 

121 

(0 

040) 

0 

121 

(0 

012) 

0 

678 

(0 

053) 

0 

237 

(0 

008) 

0 

A16 

(0 

038) 

0 

llA 

(0 

008) 

0 

831 

(0 

063) 

0 

3AA 

(0 

009) 

1 

587 

(0 

121) 

0 

282 

(0 

010) 

1 

037 

(0 

067) 

0 

120 

(0 

007  ) 

-585- 


ERIC 


631 


Table  E.16 

1986  IRT  Parameters,  Science,  Grade  3/Age  9 
Life  Sciences  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

B 

SE 

c 

SE 

N400001 

SI 

6 

1. 

053 

(0 . 

059 ) 

-0. 

250 

(0 . 

036 ) 

0. 

251 

(0 . 

oil) 

N400301 

SI 

8 

0. 

841 

(0 . 

077 ) 

0. 

164 

(0 . 

042 ) 

0. 

317 

(0 . 

oil) 

N400401 

SI 

9 

1. 

955 

(0 . 

103 ) 

-0  . 

580 

(0 . 

065  ) 

0. 

491 

(0 . 

012) 

N400402 

SI 

10 

2. 

503 

(0  . 

107 ) 

-0. 

579 

(0 . 

063 ) 

0. 

419 

(0 . 

oil) 

N400403 

SI 

11 

1. 

433 

(0 . 

072) 

-0. 

696 

(0 . 

060  ) 

0. 

517 

(0 . 

013) 

N4004O4 

SI 

12 

1. 

547 

(0 . 

092) 

-0. 

435 

(0 . 

053) 

0. 

441 

(0. 

oil) 

N40040S 

SI 

13 

1. 

669 

(0 . 

096) 

-0  . 

508 

(0  . 

059 ) 

0. 

471 

(0 . 

012) 

N400601 

SI 

17 

0. 

912 

(0 . 

061) 

-0. 

037 

(0. 

032) 

0. 

241 

(0. 

oil) 

N400701 

SI 

18 

1. 

223 

(0 . 

137 ) 

0. 

345 

(0 . 

060 ) 

0. 

280 

(0 . 

009) 

N400901 

SI 

19 

0. 

473 

(0 . 

097 ) 

2. 

237 

(0 . 

462) 

0. 

298 

(0 . 

008) 

N401001 

SI 

20 

0. 

681 

(0  . 

057 ) 

0 

480 

(0 . 

051) 

0. 

209 

(0  . 

009) 

N401101 

SI 

21 

0. 

419 

(0 . 

061) 

1 

524 

(0. 

225) 

0 

288 

(0 

010) 

N401201 

SI 

22 

1. 

202 

(0  . 

027 ) 

0 

808 

(0 

029) 

0 

265 

(0 

006) 

NA01301 

SI 

23 

0 

895 

(0 

125) 

0 

926 

(0 

142) 

0 

300 

(0 

008) 

NA12101 

SA 

10 

0 

825 

(0 

027 ) 

-0 

808 

(0 

032 ) 

0 

270 

(0 

Oil) 

N4  12201 

S4 

11 

1 

630 

(0 

051) 

-0 

481 

(0 

029) 

0 

225 

(0 

009) 

NA15501 

SA 

16 

2 

438 

(0 

076) 

1 

050 

(0 

072) 

0 

197 

(0 

004  ) 

NA12301 

SA 

21 

1 

126 

(0 

064  ) 

0 

194 

(0 

027 ) 

0 

285 

(0 

007  ) 

N413901 

S5 

5 

1 

211 

(0 

029) 

-1 

169 

(0 

039) 

0 

197 

(0 

016) 

N414101 

S5 

6 

0 

587 

(0 

025) 

-1 

322 

(0 

060 ) 

0 

218 

(0 

017 ) 

N414901 

S5 

7 

1 

274 

(0 

032) 

-0 

989 

(0 

037 ) 

0 

203 

(0 

014 ) 

N414d01 

S5 

8 

0 

527 

(0 

025) 

-r 

.814 

(0 

045) 

0 

247 

(0 

013) 

N434601 

S5 

10 

1 

123 

(0 

045) 

.476 

(0 

.  033 ) 

0 

232 

(0 

010) 

NA33001 

S5 

11 

0 

.611 

(0 

024) 

-1 

.084 

(0 

.049) 

0 

224 

(0 

.015) 

N<»15101 

S5 

lA 

1 

.  444 

(0 

.  046 ) 

-0 

.  681 

(0 

.  038 ) 

0 

.222 

(0 

.  0 12 ) 

N433101 

S5 

18 

0 

.662 

(0 

.041) 

-0 

.003 

(0 

.027) 

0 

.228 

(0 

.010) 

N415801 

S6 

6 

1 

.283 

(0 

.030) 

-1 

.158 

(0 

.039) 

0 

.209 

(0 

.016) 

N416001 

S6 

7 

0 

.588 

(0 

.032) 

-0 

.091 

(0 

.023) 

0 

.  185 

(0 

.010) 

N41S401 

S6 

11 

0 

.493 

(0 

.030) 

-0 

.219 

(0 

.027) 

0 

.253 

(0 

.011) 

N412501 

S6 

13 

0 

.996 

(0 

.067) 

0 

.112 

(0 

.031) 

0 

.204 

(0 

.009) 

NA15701 

S6 

lA 

1 

.014 

(0 

.053) 

-0 

.174 

(0 

.029) 

0 

.  193 

(0 

.010) 

N433201 

S6 

17 

0 

.893 

(0 

.109) 

0 

.833 

(0 

.111) 

0 

.212 

(0 

.007) 

NA15601 

S6 

18 

1 

.086 

(0 

.123) 

0 

.491 

(0 

.071) 

0 

.236 

(0 

.008) 

N<i37801 

S7 

10 

0 

.875 

(0 

.  129) 

0 

.903 

(0 

.  143) 

0 

.301 

(0 

.007) 

N<»37901 

S7 

11 

0 

.907 

(0 

.054) 

-0 

.005 

(0 

.029) 

0 

.218 

(0 

.  009) 

NA38101 

S7 

13 

0 

.668 

(0 

.100) 

1 

.210 

(0 

.  187) 

0 

.263 

(0 

.007  ) 

NA38701 

S7 

19 

0 

.804 

(0 

.242) 

2 

.280 

(0 

.  703) 

0 

.  255 

(C 

.006) 

N438801 

S7 

20 

0 

.  852 

(0 

.119) 

0 

.932 

(0 

.  140) 

0 

.257 

(0 

.007) 

N438901 

S7 

21 

1 

.226 

(0 

.097) 

0 

.150 

(0 

.037) 

0 

.  176 

(0 

.008) 

-586- 


Table  E.17 

1986  IRT  Parameters,  Science,  Grade  3/Age  9 
Mature  of  Science  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N403001 

S3 

12 

0 

966 

(0 

.  033 ) 

N403101 

S3 

13 

1 

039 

(0 

.  033 ) 

N403S01 

S3 

16 

0 

625 

(0 

.141) 

N403S02 

S3 

19 

0 

771 

(0 

.049) 

c  t 
oo 

20 

0 

611 

(0 

.104) 

N403701 

S3 

22 

4 

231 

(0 

.193) 

N403702 

S3 

23 

3 

314 

(0 

.152) 

N403703 

S3 

24 

3 

262 

(0 

.172) 

N403901 

S3 

29 

0 

766 

(0 

.059) 

N404001 

S3 

30 

0 

466 

(0 

.050) 

N434401 

S4 

15 

0 

667 

(0 

036) 

N413201 

S4 

17 

1 

771 

(0 

097) 

N413401 

S4 

19 

1 

690 

(0 

099) 

N413701 

S4 

23 

0 

504 

(0 

020) 

N433301 

S6 

5 

0 

602 

(0 

030) 

N433401 

S6 

6 

0 

933 

(0 

110) 

N437601 

S7 

6 

1 

146 

(0 

032) 

B 

SE 

c 

SE 

543 

(0 

.097) 

0 

.  169 

(0 

.037) 

-2 

.337 

(0 

.  067) 

0 

.  166 

(0 

.036) 

1 

.018 

( 0 

.  191 ) 

0 

.454 

(0 

.  010 ) 

-1 

147 

CO 

.064) 

0 

471 

(0 

.019) 

0 

696 

(0 

.167) 

0 

474 

(0 

.012) 

-0 

506 

(0 

.104) 

0 

33U 

(0 

.013) 

-0 

530 

(0 

.061) 

0 

374 

(0 

.013) 

-0 

461 

(0 

.061) 

0 

323 

(0 

.013) 

0 

109 

CO 

.035) 

0 

170 

(0 

.013) 

0 

746 

(0 

.064) 

0 

191 

(0 

012) 

-0 

171 

(0 

.020) 

0 

155 

(0 

009) 

0 

413 

(0 

.045) 

0 

367 

(0 

.006) 

-0 

115 

(0 

.027) 

0 

176 

(0 

.006) 

0 

221 

(0 

.017) 

0 

162 

(0 

006) 

-1 

302 

(0 

.069) 

0 

192 

(0 

019) 

0 

325 

(0 

054) 

0 

194 

(0 

010) 

-1 

964 

(0 

069) 

0 

163 

(0 

030) 

-567- 

633 


Tabl«  E.18 

1986  IRT  Parameters,  Science,  Grade  3/Age  9 
Physical  Sciences  Subscale 


FIELD  BLOCK 

ITEM 

A 

SE 

B 

SE 

c 

SE 

NA00501 

SI 

lA 

0 . 

516 

(0. 

101) 

2. 

456 

(0. 

493) 

0  . 

354 

(0. 

015) 

N400101 

SI 

15 

0  . 

638 

(0. 

174) 

2. 

898 

(0. 

831) 

0. 

500 

(0  . 

015) 

N400102 

SI 

16 

0. 

680 

(0. 

177) 

2. 

828 

(0. 

784  ) 

0  . 

471 

(0  . 

015 ) 

N401501 

S2 

1 

0  . 

345 

(0. 

050) 

0. 

855 

(0. 

133) 

0  . 

349 

(0  . 

018) 

N^01702 

S2 

0  . 

456 

(0. 

086) 

2. 

169 

(0. 

420) 

0  . 

522 

(0. 

015) 

N401703 

S2 

5 

0  . 

444 

(0. 

083) 

2. 

174 

(0. 

418) 

0. 

492 

(0  . 

015) 

NA01801 

S2 

6 

0. 

742 

(0. 

102) 

0. 

352 

(0. 

084 ) 

0  . 

462 

(0  . 

021 ) 

NA01802 

S2 

7 

0  . 

592 

(0. 

085) 

-0. 

113 

(0. 

061) 

0  . 

485 

(0. 

022) 

N401803 

S2 

8 

0. 

593 

(0. 

083) 

0. 

749 

(0. 

128) 

0  . 

500 

(0. 

019) 

N401804 

S2 

9 

0  . 

572 

(0. 

090) 

2. 

257 

(0. 

376) 

0. 

469 

(0. 

015) 

NA01901 

S2 

10 

0. 

514 

(0. 

083) 

2. 

637 

(0. 

440) 

0  . 

329 

(0. 

015) 

NA02001 

S2 

11 

1 . 

047 

(0. 

132) 

0. 

336 

(0. 

093) 

0  . 

462 

(0. 

020 ) 

NA02002 

S2 

12 

1 . 

123 

(0. 

142) 

0. 

407 

(0. 

101) 

0. 

489 

(0. 

019) 

N402003 

S2 

13 

0  . 

511 

(0. 

192) 

4  . 

930 

(1. 

918) 

0  . 

460 

(0. 

013  ) 

NA02005 

S2 

15 

0 . 

852 

(0. 

101) 

0. 

931 

(0. 

144) 

0  . 

481 

(0  . 

018) 

N402201 

S2 

17 

0  . 

830 

(0. 

062) 

1. 

066 

(0. 

096) 

0  . 

171 

(0. 

014  ) 

NA02602 

S2 

21 

0 . 

589 

(0. 

075) 

0. 

757 

(0. 

117) 

0 

471 

(0 

018) 

N402603 

S2 

22 

0 

423 

(0. 

144) 

5 

053 

(1 

783) 

0 

486 

(0 

013) 

N^02701 

S2 

23 

0 

559 

(0 

077) 

2 

741 

(0 

393) 

0 

207 

(0 

013 ) 

NA02801 

S2 

24 

0 

590 

(0 

193) 

4 

814 

(1 

638) 

0 

246 

(0 

010 ) 

N402901 

S2 

25 

0 

517 

(0 

162) 

5 

032 

(1 

610) 

0 

179 

(0 

Oil) 

N403201 

S3 

14 

1 

678 

(0 

160) 

-0 

556 

(0 

087  ) 

0 

232 

(0 

029) 

N403202 

S3 

15 

1 

093 

(0 

088) 

-0 

186 

(0 

047  ) 

0 

145 

(0 

021 ) 

N403301 

S3 

16 

0 

614 

(0 

067) 

0 

341 

(0 

059) 

0 

238 

(0 

020 ) 

N403401 

S3 

17 

0 

776 

(0 

073) 

1 

819 

(0 

191) 

0 

292 

(0 

012 ) 

N^03601 

S3 

21 

0 

.948 

(0 

080) 

1 

.484 

(0 

.  153) 

0 

.286 

(0 

.  014 ) 

N403801 

S3 

25 

0 

.574 

(0 

.090) 

1 

.981 

(0 

.  328) 

0 

.453 

(0 

.  016) 

HA03802 

S3 

26 

0 

.520 

(0 

.  164) 

4 

.  153 

(1 

.350) 

0 

.415 

(0 

.  014 ) 

N403803 

S3 

27 

0 

.772 

(0 

.110) 

0 

.518 

(0 

.  108) 

0 

.516 

(0 

.  019) 

N403804 

S3 

?.8 

0 

.781 

(0 

.  109) 

0 

.894 

(0 

.  153 ) 

0 

.486 

( 0 

.018) 

N40A201 

S3 

31 

0 

.647 

(0 

.081) 

2 

.413 

(0 

.319) 

0 

.204 

(0 

.013) 

NA13601 

S^ 

12 

0 

.657 

(0 

.070) 

3 

.877 

(0 

.421) 

0 

.0 

(0 

.0  ) 

NA12801 

S^ 

14 

0 

.865 

(0 

.080) 

0 

.756 

(0 

.087  ) 

0 

.207 

(0 

.015) 

N^33601 

S^ 

20 

0 

.574 

(0 

.060) 

1 

.656 

(0 

.182) 

0 

.  181 

(0 

.013) 

N^12701 

S^ 

22 

0 

.791 

(0 

.071) 

2 

.035 

(0 

.200) 

0 

143 

(0 

.011) 

N^3^501 

S5 

9 

1 

.153 

(0 

.058) 

1 

.371 

(0 

.086) 

0 

.0 

(0 

.0  ) 

N^33501 

S6 

9 

0 

.424 

(0 

.058) 

-0 

.599 

(0 

.091) 

0 

.339 

(0 

.023) 

N^16701 

S6 

15 

0 

.867 

(0 

.089) 

0 

.089 

(0 

.033) 

0 

.0 

(0 

.0  ) 

N^37701 

S7 

9 

1 

.009 

(0 

.078) 

1 

.977 

(0 

.184) 

0 

.194 

(0 

.010) 

N438001 

S7 

12 

0 

.592 

(0 

.070) 

1 

.583 

(0 

.175) 

0 

.292 

(0 

.013) 

NA33201 

S7 

14 

1 

.023 

(0 

.096) 

0 

.432 

(0 

.065) 

0 

.  181 

(0 

.016) 

N^38301 

S7 

15 

0 

.460 

(0 

.052) 

0 

.763 

(0 

.095) 

0 

.204 

(0 

.017) 

NA38401 

S7 

16 

0 

.664 

(0 

.081) 

2 

.426 

(0 

.312) 

0 

.251 

(0 

.011) 

NA38501 

S7 

17 

0 

.902 

(0 

.079) 

1 

.929 

(0 

.195) 

0 

.241 

(0 

.011) 

ERIC 


-588- 

63. 


Table  £.19 

1986  IRT  Parameters,  Science,  Grade  7/Ase  13 
Life  Sciences  Subscale 


FIELD 

BLOCK 

ITEM 

NA0A501 

SI 

12 

Q 

N404601 

1^  "t  U  "t      U  X 

SI 

13 

Q 

NA0A701 

SI 

14 

Q 

NA0A702 

SI 

15 

Q 

NA00201 

SI 

16 

0 

NA05001 

SI 

23 

0 

NA05201 

SI 

25 

0 

N401201 

SI 

28 

1 

NA05601 

SI 

30 

0 

NA05701 

SI 

31 

1 

NA05801 

SI 

32 

0 

N405901 

SI 

33 

0 

N406001 

SI 

34 

1 

NA06101 

SI 

35 

1 

NA06201 

SI 

36 

1 

NA12101 

S4 

10 

0 

NA12201 

S4 

11 

1 

NA 1550 1 

S4 

1 6 

2 

^  X  ^  w  U  X 

S4 

21 

1 

N4 19*501 

S5 

3 

u 

N4  1  Q7C\^ 

~  X  9  ^  U  X 

S5 

1 

H4  1  Q')  n  1 

^  X  9  <^  U  X 

S5 

/ 

u 

N4 1940 1 

S5 

g 

1 

X 

N420 101 

10 

Q 

N4 19 10 1 

41  ^  X  ^  X  U  X 

S5 

13 

1 

N418401 

S6 

11 

1 

N418301 

S6 

12 

1 

N418201 

S6 

17 

1 

N418501 

S6 

20 

0 

N418101 

S6 

22 

N417101 

S6 

27 

1 

N421101 

S7 

11 

0 

K421301 

S7 

14 

0 

N421302 

S7 

15 

1 

N421401 

S7 

17 

0 

N433701 

S7 

18 

0 

N421201 

S7 

21 

0 

N423401 

S8 

15 

1 

N423501 

S8 

18 

1 

N423301 

S8 

23 

0 

N423601 

S8 

27 

0 

N435801 

S9 

22 

1 

N436301 

S9 

27 

1 

N436601 

S9 

31 

1 

A  SE  B 


890 

( 0 

047 ) 

1 

LtXL 

676 

( 0 

024  ) 

u 

900 

( 0 

056 ) 

u 

D03 

802 

( 0 

052 ) 

Q 

336 

769 

( 0 

033 ) 

-0 

292 

474 

( 0 

019) 

u 

A  7  A 

391 

( 0 

020 ) 

Q 

381 

202 

( 0 

027 ) 

u 

AHA 

ou  o 

321 

( 0 

042 ) 

7  Q^a 
too 

454 

( 0 

063 ) 

u 

Aft7 
o<j  / 

820 

( 0 

051 ) 

ZD  D 

895 

( 0 

075 ) 

Q7  n 

57  7 

( 0 

034 ) 

z 

U  DO 

184 

( 0 

044 ) 

Z 

DD  u 

138 

( 0 

030 ) 

2 

.  438 

825 

( 0 

027  ) 

u 

Ann 
ou  o 

OOKJ 

f  n 

V  ^ 

u 

A  A  1 

HOO 

f  n 

\  w 

n  7  R  ^ 

1 

050 

1  9  K 

UpH  J 

0 

194 

OOO 

( 0 

031) 

-0 

.814 

OH  J 

U  H  o  J 

~0 

268 

Q  1  Q 

(  0 

023  ) 

0 

437 

u  xo 

n  9  T  ^ 

1 

335 

UZ  1  J 

0 

903 

1  9Q 

n  9  K  ^ 

U  Z  D  J 

0 

83 1 

u  /  o 

n  1 7  ^ 

0 

010 

064 

(  0 

041) 

u 

XZH 

288 

(  0 

031 ) 

1 

X 

H  y  y 

581 

(  0 

021 ) 

1 

743 

908 

(0 

025) 

1 

490 

286 

(0 

033) 

2 

117 

503 

(0 

042) 

-0 

409 

974 

(0 

053) 

0 

580 

184 

(0 

072) 

0 

233 

575 

(0 

043) 

0 

136 

738 

(0 

051) 

0 

961 

915 

(0 

060) 

1 

270 

301 

(0 

074) 

-0 

623 

048 

(0 

073) 

-0 

029 

978 

(0 

059) 

0 

849 

854 

(0 

066) 

1 

581 

336 

(0 

026) 

1 

064 

347 

(0 

077  ) 

1 

111 

166 

(0 

130) 

2 

287 

SE  C  SE 


n  A  ^  ^ 

U  O  D  J 

U 

17  9 

X  /z 

( 0 

028 ) 

( 0 

028 ) 

U 

9  A  A 
ZHO 

f  r\ 

U  xU ) 

( 0 

049) 

U 

99  'a 
ZZO 

f  r\ 

U  X9  ) 

n  'a  ^  ^ 

U  O  D  J 

U 

9  9^ 
ZZD 

f  r\ 

012) 

( 0 

023 ) 

M 

9*99 
ZOZ 

f  r\ 

U  lo  ) 

( 0 

025 ) 

u 

90  ^ 
ZU  D 

f  n 

U  X  U  ) 

( 0 

026 ) 

u 

917 
Z  X  / 

f  n 

U  XU  J 

( 0 

029 ) 

u 

9R 
ZD  D 

f  n 

n  n  c  ^ 
U  U  D  ) 

( 0 

236 ) 

u 

90  7 
ZU  / 

^  n 
\  U 

U  XU  J 

( 0 

059 ) 

u 

1  OA 
XoO 

009) 

/  0 

086 ) 

G 

1  R  1 
XO  X 

(  0 

009) 

( 0 

18 1 ) 

u 

1  Q  A 

xy  O 

\  u 

u  U  O  J 

/  0 

071) 

u 

9  1  O 

Z  X9 

(  0 

.005) 

( 0 

122 ) 

u 

9 

Z  DD 

(  0 

.005) 

/  0 

080 ) 

u 

X  X  D 

(  0 

004) 

/  0 

032 ) 

u 

9  7  n 
Z  /  U 

(  0 

Oil) 

u  z  y  J 

0 

225 

(  0 

009 ) 

072) 

0 

197 

(  0 

004 ) 

n  o  7  ^ 
UZ  /  J 

0 

285 

( 0 

007 ) 

(  0 

041) 

0 

236 

( 0 

015) 

(  0 

024  ) 

C 

152 

( 0 

013) 

( 0 

021 ) 

0 

14 1 

( 0 

009 ) 

n  T  Q  ^ 

0 

219 

( 0 

006 ) 

(  0 

030 ) 

0 

163 

( 0 

007  ) 

uzy  J 

0 

233 

( 0 

007 ) 

n  9  n  ^ 

u  Z  U  J 

0 

199 

( 0 

Oil) 

(  0 

020  ) 

u 

1  QQ 

xy  9 

( 0 

0 12 ) 

(  0 

053  ) 

u 

OD  X 

( 0 

006) 

(  0 

068) 

0 

216 

( 0 

006 ) 

(0 

049) 

0 

267 

(0 

006) 

(0 

075) 

0 

217 

(0 

005) 

(0 

042) 

0 

236 

(0 

015) 

(0 

044) 

0 

192 

(0 

010) 

(0 

033) 

0 

.184 

(0 

Oil) 

(0 

026) 

0 

206 

(0 

013) 

(0 

073) 

0 

216 

(0 

010) 

(0 

095) 

0 

277 

(0 

008) 

(0 

048) 

0 

217 

(0 

019) 

(0 

028) 

0 

215 

(0 

.013) 

(0 

062) 

0 

232 

(0 

009) 

(0 

133) 

0 

246 

(0 

008) 

(0 

032) 

0 

179 

CO 

006) 

(0 

086) 

0 

288 

(0 

008) 

(0 

303) 

0 

232 

(0 

007) 

-589- 

633 


Table  E.20 

1986  IRT  Parameters,  Science,  Grade  7/Age  13 
Chemistry  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N404901 

SI 

17 

1 

136 

(0. 

076) 

N404902 

SI 

18 

0 

567 

(0. 

213) 

N404801 

SI 

20 

1 

057 

(0. 

060) 

N404802 

SI 

21 

1 

721 

(0. 

136) 

1 »  ■?  w  ■?      w  o 

SI 

22 

184 

(0 

126) 

11  "4  U  J  X  U  X 

SI 

2A 

Q 

946 

(0 

039) 

N405301 

SI 

26 

1 

0  53 

(0 

111) 

N405401 

SI 

27 

0 

720 

(0 

032) 

N405501 

SI 

29 

0 

613 

(0 

034 ) 

N419801 

S5 

2 

1 

135 

(0 

068) 

NA18701 

S5 

11 

1 

236 

(0 

041) 

N418702 

S5 

12 

0 

508 

(0 

027) 

NA20201 

S5 

1<» 

1 

120 

(0 

043) 

N420001 

S5 

15 

0 

849 

(0 

032) 

N419701 

S5 

16 

0 

.592 

(0 

031) 

N419901 

S5 

17 

0 

.473 

(0 

028) 

N419601 

S5 

19 

1 

.  105 

(0 

047) 

NA23101 

S8 

17 

0 

.702 

(0 

057) 

N423001 

S8 

21 

0 

.547 

(0 

.057) 

N423201 

S8 

24 

0 

.828 

(0 

.081) 

N422901 

S8 

29 

0 

.789 

(0 

.097) 

N422801 

S8 

30 

1 

.  124 

(0 

.  123) 

N436201 

S9 

26 

0 

.729 

(0 

.040) 

B  SE  C  SE 


-0  . 

317 

( 0  . 

043 ) 

0  . 

259 

(0  . 

015 ) 

4 . 

197 

( 1 . 

655 ) 

0 

432 

(0  . 

008 ) 

-1 . 

624 

(0 . 

111 ) 

0  . 

458 

( 0 . 

033 ) 

-0  . 

363 

( 0 . 

062 ) 

0 

330 

(0 

016 ) 

0  . 

248 

( 0 

057 ) 

0 

341 

(0 

012 ) 

0 

80  7 

(0 

047 ) 

0 

239 

(0 

008 ) 

0 

900 

(0 

112 ) 

0 

216 

( 0 

010 ) 

1 

069 

(0 

055) 

0 

166 

f  0 

008) 

0 

267 

(0 

029) 

0 

197 

(0 

012) 

-0 

433 

(0 

039) 

0 

319 

(0 

014) 

1 

109 

(0 

054) 

0 

162 

(0 

006) 

1 

461 

(0 

081) 

0 

148 

(0 

007) 

1 

244 

(0 

067) 

0 

277 

(0 

007) 

1 

135 

(0 

051) 

0 

112 

(0 

006) 

1 

451 

(0 

081) 

0 

186 

(0 

007) 

1 

195 

(0 

074) 

0 

187 

(0 

008) 

1 

515 

(0 

088) 

0 

356 

(0 

007) 

-0 

211 

(0 

034) 

0 

.193 

(0 

015) 

0 

199 

(0 

037) 

0 

.211 

(0 

013) 

1 

.792 

(0 

.181) 

0 

.0 

(0 

.0  ) 

0 

.657 

(0 

.091) 

0 

.225 

(0 

.011) 

0 

.315 

(0 

.053) 

0 

.155 

(0 

.011) 

1 

.253 

(0 

.075) 

0 

.  171 

(0 

.007) 

590- 


TabI*  E.21 

1986  IRT  Param«t«ri,  Sci«nc«,  Grada  13 
Nature  of  Sci«nc«  Subscal* 


i  I  ELD 

BLOCK 

ITEM 

N408301 

S3 

10 

N408302 

S3 

11 

N4  08303 

S3 

12 

H408304 

S3 

13 

N408401 

S3 

14 

N408501 

S3 

15 

N408502 

S3 

16 

N408601 

S3 

17 

N408701 

S3 

18 

N408801 

S3 

19 

N408901 

S3 

20 

N408902 

S3 

21 

N408903 

S3 

22 

N408904 

S3 

23 

N409001 

S3 

24 

N409101 

S3 

25 

N409102 

S3 

26 

N409103 

S3 

27 

N409201 

S3 

28 

N409301 

S3 

29 

N4  0950 1 

S3 

33 

N409601 

S3 

34 

N409701 

S3 

35 

N434401 

S4 

15 

N413201 

S4 

17 

N413401 

S4 

19 

N435001 

S4 

23 

N434901 

S4 

27 

N435501 

S9 

15 

N413701 

S9 

18 

N436001 

S9 

24 

N436401 

S9 

28 

N436501 

S9 

29 

ERIC 


A 

SE 

B 

SE 

c 

SE 

2 

513 

(0 

.  057 ) 

0 

.  800 

0 

.358 

( 0 

.  007 ) 

0 

773 

(0 

.  043 ) 

-0 

.  458 

f  n 

.  \J09  ) 

0 

.404 

( 0 

.  016 ) 

0 

920 

(0 

.  045 ) 

-0 

.  574 

( 0 

.  040 ) 

0 

.410 

( 0 

.016) 

0 

946 

(0 

.  054 ) 

-0 

.362 

n')  A  ^ 

,  UOO  ) 

0 

.424 

( 0 

.  016 ) 

0 

672 

(0 

.  049 ) 

0 

.260 

f  n 

.  UO^  ) 

0 

.221 

( 0 

.012) 

0 

963 

( 0 

.  052 ) 

-0 

.676 

f  n 

.  UHO) 

0 

205 

( 0 

.  019 ) 

0 

614 

( 0 

.  045 ) 

1 

.060 

f  n 

.  Uoo  ; 

0 

156 

( 0 

.  009 ) 

0 

462 

(0 

.  024 ) 

-0 

.376 

f  n 

.  UZD  ) 

0 

196 

( 0 

.  012 ) 

0 

512 

( 0 

.  043 ) 

0 

.458 

f  n 

n  A  7  ^ 

0 

233 

(  0 

.  012 ) 

0 

634 

( 0 

.  025 ) 

0 

460 

(  0 

not  \ 
.  UZ/  ; 

0 

195 

( 0 

.  010 ) 

1 

444 

( 0 

.  060 ) 

0 

380 

n  A  n  \ 

0 

521 

( 0 

.  009 ) 

1 

553 

( 0 

.  104 ) 

-0 

554 

f  n 

n ^ 

.  \JD  /  ) 

0 

516 

( 0 

.016) 

1 

003 

( 0 

.  039 ) 

0 

563 

f  n 

n  'a  7  ^ 

0 

419 

( 0 

.  009 ) 

1 

21E 

( 0 

.  043 ) 

0 

928 

f  r\ 
\  w 

U  J 

0 

496 

( 0 

008 ) 

0 

8C  J 

(C 

.  047 ) 

0 

283 

f  n 

UZa  ) 

0 

137 

( 0 

Oil ) 

1 

230 

( 0 

.078) 

-0 

239 

(  0 

035  ) 

0 

299 

( 0 

015 ) 

0 

987 

/  n 

.  U  D  X  J 

0 

618 

(  0 

051 ) 

0 

258 

(  0 

010 ) 

0 

798 

f  0 

.  081 ) 

1 

871 

(  0 

205  ) 

0 

342 

( 0 

008) 

1 

082 

(0 

073) 

0 

880 

(0 

077  ) 

0 

328 

( 0 

010 ) 

1 

362 

(0 

052) 

0 

232 

(0 

027) 

0 

167 

(0 

010) 

u 

O  A  *5 

(0 

026) 

1 

657 

(0 

060) 

0 

128 

(0 

006) 

1 

471 

(0 

089) 

1 

349 

(0 

119) 

0 

296 

(0 

008) 

0 

653 

(0 

077) 

2 

262 

(0 

275) 

0 

144 

(0 

008) 

0 

887 

(0 

038) 

-0 

171 

(0 

020) 

0 

155 

(0 

009) 

1 

771 

(0 

097) 

0 

413 

(0 

045) 

0 

367 

(0 

006) 

1. 

690 

(0 

099) 

-0 

115 

(0 

027) 

0 

178 

CO 

008) 

0. 

905 

(0 

055) 

0 

658 

(0 

049) 

0 

175 

(0 

010) 

0. 

916 

(0 

059) 

0 

922 

(0 

070) 

0 

248 

CO 

009) 

1. 

213 

(0 

066) 

-0 

717 

(0 

050) 

0 

197 

(0 

020) 

0. 

504 

(0 

020) 

0 

221 

(0 

017) 

0. 

162 

(0 

008) 

0. 

728 

(0 

065) 

0 

390 

(0 

045) 

0. 

253 

(0 

Oil) 

1. 

399 

(0 

061) 

2 

287 

(0 

145) 

0. 

359 

(0 

005) 

1. 

895 

(0 

045) 

0. 

678 

(0 

035) 

0. 

135 

(0 

007) 

-591- 

637 


Table  E.22 

1986  IRT  Parameters,  Science,  Grade  7/A5e  13 
Physics  Subscale 


FIELD      BLOCK  ITEM 


NA067  01 

S2 

2 1 

NA0690 1 

S2 

28 

NhO/O  01 

S2 

O  Q 

NA  07 10 1 

S2 

oU 

NA07201 

S2 

3 1 

NA08001 

S2 

3  A 

NA07601 

S2 

3  5 

NA07901 

S2 

39 

NA13601 

S4 

12 

NA 12801 

S4 

lA 

N43360 1 

SA 

20 

N4  12701 

S4 

22 

N412601 

SA 

2  A 

N421801 

S7 

10 

N421901 

S7 

16 

N421701 

S7 

2A 

N422001 

S7 

26 

N421501 

S7 

27 

N422101 

S8 

16 

N422201 

S8 

19 

NA22501 

S8 

22 

N422301 

S8 

25 

N422401 

S8 

26 

N423701 

S8 

28 

N421601 

S8 

32 

N435401 

S9 

lA 

N436701 

S9 

17 

N435701 

S9 

21 

N435901 

S9 

23 

N436107 

S9 

25 

ERIC 


SE 

B 

SE 

c 

SE 

0  . 

617 

(0. 

055) 

-0  . 

222 

(0. 

037) 

0  . 

197 

(0. 

016) 

0  . 

550 

(0. 

033) 

0  . 

017 

(0. 

022) 

0. 

195 

(0. 

012) 

0  . 

368 

(0. 

028) 

0  . 

2A1 

(0. 

029) 

0. 

219 

(0. 

012) 

1 . 

152 

(0. 

038) 

2 . 

019 

(0. 

091) 

0. 

160 

(0. 

006) 

0  . 

555 

(0. 

032) 

0  . 

762 

(0. 

050) 

0. 

219 

(0. 

009) 

1  ^ 

138 

(0. 

llA) 

0  . 

889 

(0. 

109) 

0  . 

223 

(0. 

010) 

0  . 

890 

(0. 

103) 

0  . 

687 

(0. 

091) 

0 . 

163 

(0. 

Oil) 

Q  ^ 

7  A  8 

(0. 

107) 

1 . 

A8A 

(0. 

225) 

0  . 

2A1 

(0. 

Oil) 

0  . 

627 

(0. 

070) 

1 . 

888 

(0. 

212) 

0 . 

0 

(0. 

0  ) 

2^ 

067 

(0. 

053) 

-0  . 

935 

(0. 

060  ) 

0  . 

191 

(0. 

021) 

Q  ^ 

AO  1 

(0. 

0A9) 

0 

377 

(0. 

05A) 

0 . 

200 

(0. 

OlA) 

0 

603 

(0 

075) 

0 

692 

(0. 

093) 

0 

195 

(0 

Oil) 

0 

A8  A 

(0 

136) 

3 

589 

(1 

013) 

0 

162 

(0 

007) 

Q 

A66 

(0 

OAO) 

-1 

399 

(0 

123) 

0 

203 

(0 

022) 

Q 

681 

(0 

0A5) 

-0 

076 

(0 

023) 

0 

217 

(0 

012) 

Q 

350 

(0 

037) 

2 

919 

(0 

308) 

0 

220 

(0 

007 ) 

0 

6A1 

(0 

113) 

2 

069 

(0 

372) 

0 

235 

(0 

009) 

0 

512 

(0 

.127) 

3 

.132 

(0 

7  82) 

0 

.239 

(0 

.008) 

0 

.378 

(0 

.029) 

-0 

.907 

(0 

.072) 

0 

.209 

(0 

.015) 

0 

.6A1 

(0 

.OAO) 

0 

.3A7 

(0 

.031) 

0 

.  195 

(0 

.010) 

0 

.889 

(0 

.  115) 

0 

.5A9 

(0 

.083) 

0 

.216 

(0 

.011) 

0 

.506 

(0 

.069) 

1 

.162 

(0 

.  163) 

0 

.  175 

(0 

.011) 

0 

.666 

(0 

.0A7) 

2 

.305 

(0 

.  173) 

0 

.285 

(0 

.007) 

0 

.879 

(0 

.  105) 

1 

.306 

(0 

.  168) 

0 

.  183 

(0 

.009) 

0 

.13A 

(0 

.029) 

6 

.522 

(1 

.395) 

0 

.202 

(0 

.007) 

0 

.86A 

(0 

.050) 

-0 

.113 

(0 

.037  ) 

0 

.  185 

(0 

.OlA) 

0 

.525 

(0 

.055) 

0 

.llA 

(0 

.033) 

0 

.225 

(0 

.OlA) 

0 

.521 

(0 

.060) 

0 

.7A9 

(0 

.092) 

0 

.  197 

(0 

.012) 

1 

.15A 

(0 

.0A6) 

2 

.079 

(0 

.  106) 

0 

.  lAA 

(0 

.005) 

0 

.872 

(0 

.095) 

1 

.llA 

(0 

.  132) 

0 

.  177 

(0 

.009) 

-592- 

63 


Table  E.23 

1986  IRT  Parameters,   Science,  Grade  7/Age  13 
Earth  and  Space  "Science  Subscale 


r  ICiLU 

BLOCK 

ITEM 

NAO  630 1 

S2 

10 

0 

NAO  6302 

S2 

11 

0 

U  A  n  C  O  A  o 

NAO  6303 

S2 

12 

0 

S2 

13 

0 

HAO  6*1  0  1 

S2 

lA 

0 

u  A  n  c  A  n  o 
NAO  6*1  OZ 

S2 

IS 

1 

V  L  e\  a  L  e\ 

oZ 

16 

0 

oZ 

17 

1 

M  A  n  A  A  n  <; 

£1*4  U  o*4  U  D 

CO 

oZ 

18 

0 

w  A  n  c  c  n  1 

S2 

19 

0 

M  /,  n  c  c  n  1 
flA  U  bb  U 1 

S2 

20 

0 

n  M  u  o  o  u  X 

c  o 
oZ 

22 

1 

MA  r»  R  ft  r»  o 
Mm  u  bo  uz 

c  o 
oZ 

23 

1 

MA  r»  A  ft  r»  'J 
Mm  u  bo 

S2 

24 

0 

MA  nf^ftriA 

M*4  U  D  O  U  *» 

c  o 
oZ 

25 

0 

MA  nfionK 

M*4  U  bo  U  D 

S2 

26 

1 

N AOboOb 

S2 

27 

0 

U  #  A          A  1 

MAO  730 1 

S2 

32 

0 

M  A  A  TJ  A  O 

MAO / o  OZ 

S2 

33 

0 

U  #  A  T  T  A  1 

MAO / / 0 1 

S2 

37 

0 

\S  1  A  T  O  A  1 

MAO  /  oO 1 

S2 

38 

0 

MA  0820 1 

S2 

AO 

1 

MA 12901 

S  A 

13 

0 

MAI  C  A  A 1 

MAI bA  0 1 

S  A 

26 

0 

*T  #  O  CO  A  1 

NA3D201 

S5 

6 

0 

NA 1760 1 

S6 

13 

1 

M  A  1  Q  A  A  1 

MA loO  0  1 

S6 

15 

0 

M  A  1  *\  1  A  1 
M*4  O  Dl  U  1 

S6 

16 

0 

MA  1  7fln  1 

S6 

1  Q 

1 

NA17701 

S6 

21 

1 

NAlAAOl 

S6 

23 

0 

NA16801 

S6 

25 

1 

NA17901 

S6 

26 

1 

NA20601 

S7 

13 

0 

NA20A01 

S7 

19 

0 

NA20501 

S7 

20 

1 

NA20701 

S7 

22 

0 

NA20301 

S7 
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(0. 

047) 

0. 

239 

(0. 

008) 

1. 

069 

(0. 

055) 

0 

166 

(0. 

008) 

4. 

437 

(0 

716) 

0 

127 

(0 

009) 

0. 

267 

(0. 

029) 

0 

197 

(0 

012) 

1. 

026 

(0 

075) 

0 

225 

(0 

015) 

1. 

035 

(0 

078) 

0 

193 

(0 

012) 

1. 

546 

(0 

087) 

0 

208 

(0 

012) 

1. 

779 

(0. 

092) 

0 

150 

(0 

Oil) 

1. 

104 

(0 

072) 

0 

173 

(0 

012) 

2 

459 

(0 

171) 

0 

237 

(0 

012) 

-0 

727 

(0 

099) 

0 

212 

(0 

023) 

1 

902 

(0 

163) 

0 

219 

(0 

012) 

1 

713 

(0 

183) 

0 

220 

(0 

014) 

1 

767 

(0 

161) 

0 

250 

(0 

013) 

-0 

433 

(0 

039) 

0 

319 

(0 

014) 

1 

109 

(0 

054) 

0 

162 

(0 

006) 

1 

461 

(0 

081) 

0 

148 

(0 

007) 

1 

244 

(0 

067) 

0 

277 

(0 

007) 

1 

135 

(0 

051) 

0 

112 

(0 

006) 

1 

451 

(0 

081) 

0 

186 

(0 

007) 

1 

195 

(0 

074) 

0 

187 

(0 

008) 

1 

515 

(0 

088) 

0 

356 

(0 

007) 

-0 

241 

(0 

046) 

0 

222 

(0 

020) 

-0 

199 

(0 

041) 

0 

214 

(0 

022) 

1 

854 

(0 

133) 

0 

210 

(0 

009) 

2 

179 

(0 

145) 

0 

113 

(0 

007) 

2 

661 

(0 

189) 

0 

102 

(0 

006) 

2 

036 

(0 

134) 

0 

168 

(0 

009) 

1 

949 

(0 

139) 

0 

238 

(0 

Oil) 

2 

362 

(0 

182) 

0 

272 

(0 

Oil) 

0 

779 

(0 

068) 

0 

.263 

(0 

015) 

2 

195 

(0 

183) 

0 

.345 

(0 

012) 

2 

805 

(0 

275) 

0 

.284 

(0 

009) 

3 

112 

(0 

281) 

0 

.205 

(0 

.008) 

-0 

167 

(0 

035) 

0 

.220 

(0 

021) 

0 

716 

(0 

060) 

0 

.212 

(0 

016) 

1 

035 

(0 

080) 

0 

.228 

(0 

015) 

2 

877 

(0 

130) 

0 

.0 

(0 

.0  ) 

1 

253 

(0 

075) 

0 

.  171 

(0 

.007) 

2 

.847 

(0 

.207) 

0 

.  137 

(0 

.009) 

1 

.822 

(0 

.  108) 

0 

.  138 

(0 

.011) 

2 

.465 

(0 

.  182) 

0 

.207 

(0 

.009) 

-595- 


641 


Table  £.26 

1986  IRT  Parameters,  Science.  Grade  ll/Age  17 
Nature  of  Science  Subscale 


FIELD 

BLOCK 

ITEM 

A 

SE 

N408601 

SI 

19 

0. 

462 

( 0  . 

024 ) 

N409301 

SI 

20 

1. 

362 

(0 . 

052 ) 

N410101 

SI 

25 

0. 

707 

(0 . 

04  8) 

N410102 

SI 

26 

0. 

432 

(0 . 

037 ) 

N410103 

SI 

27 

0. 

607 

(0. 

051 ) 

N409S01 

SI 

34 

0. 

842 

(0  . 

026) 

N408301 

S3 

10 

2. 

513 

(0  . 

057 ) 

N408302 

S3 

11 

0. 

773 

(0. 

043 ) 

N408303 

S3 

12 

0. 

920 

(0 . 

045) 

NA0830A 

S3 

13 

0. 

946 

(0  . 

054 ) 

NA08901 

S3 

15 

1. 

444 

(0 . 

060 ) 

NA08902 

S3 

16 

1 

553 

( 0 . 

104 ) 

NA08903 

S3 

17 

1 

003 

( 0 

039 ) 

K408904 

S3 

18 

1 

218 

( 0 

04  3) 

NA11201 

S3 

23 

0 

730 

(0 

030 ) 

NA08801 

S3 

24 

0 

634 

(0 

025 ) 

NA11501 

S3 

26 

1 

666 

(0 

042 ) 

NA11502 

S3 

27 

0 

764 

(0 

04  4) 

NA28901 

S7 

24 

0 

832 

\  u 

032 ) 

NA28801 

S7 

28 

1 

093 

( 0 

034  ) 

NA3A001 

S7 

29 

1 

073 

( 0 

079 ) 

NA28601 

S7 

30 

1 

146 

033 ) 

NA29001 

S7 

37 

1 

007 

(0 

048) 

NA31901 

S8 

17 

1 

.  178 

(0 

049) 

NA31902 

S8 

18 

0 

.870 

(0 

.029) 

NA32A01 

S8 

25 

1 

.226 

(0 

.042) 

NA25201 

S9 

26 

0 

.  827 

(0 

.032) 

NA25301 

S9 

30 

0 

.674 

(0 

.031) 

NA25A01 

S9 

33 

1 

.  196 

(0 

.067) 

NA25901 

SIO 

22 

0 

.  724 

(0 

.032) 

NA25801 

SIO 

25 

0 

.680 

(0 

.035) 

NA13701 

Sll 

12 

0 

.504 

(0 

.020) 

NA36901 

Sll 

15 

0 

.996 

(0 

.031) 

NA36501 

Sll 

21 

1 

.895 

(0 

.045) 

NA36A01 

Sll 

22 

1 

.399 

(0 

.051) 

NA32001 

Sll 

28 

1 

.380 

(0 

.045) 

B  SE  C  SE 


-0. 

376 

(0. 

026) 

0. 

196 

(0. 

012) 

0. 

232 

(0. 

027) 

0. 

167 

(0. 

010) 

0. 

272 

(0. 

045) 

0. 

489 

(0. 

018) 

0. 

674 

(0. 

069) 

0. 

476 

(0. 

016) 

-0. 

401 

(0. 

053) 

0. 

461 

(0. 

023) 

1. 

657 

(0. 

060) 

0. 

128 

(0. 

006) 

0. 

800 

(0. 

050) 

0. 

358 

(0. 

007  ) 

-0. 

458 

(0. 

039) 

0  . 

404 

(0. 

016) 

-0. 

574 

(0 

040) 

0. 

410 

(0. 

016) 

-0. 

362 

(0 

038) 

0. 

424 

(0. 

016) 

0. 

380 

(0 

040) 

0. 

521 

(0. 

009) 

-0. 

554 

(0 

057) 

0. 

516 

(0. 

016) 

0 

563 

(0 

037  ) 

0 

419 

(0 

009) 

0 

928 

(0 

052) 

0 

496 

(0 

008) 

1 

372 

(0 

067) 

0 

251 

(0 

012) 

0 

460 

(0 

027) 

0 

195 

(0 

010) 

1 

928 

(0 

084) 

0 

187 

(0 

008) 

0 

110 

(0 

037  ) 

0 

314 

(0 

021) 

1 

234 

(0 

059) 

0 

236 

(0 

012) 

1 

201 

(0 

053) 

0 

199 

(0 

Oil) 

2 

944 

(0 

.271) 

0 

373 

(0 

008) 

1 

640 

(0 

.064) 

0 

166 

(0 

009) 

2 

996 

(0 

.171) 

0 

094 

(0 

006) 

0 

.395 

(0 

.037) 

0 

.181 

(0 

.017) 

1 

.964 

(0 

.076) 

0 

.  114 

(0 

.008) 

0 

.650 

(0 

.041) 

0 

.  164 

(0 

.015) 

1 

.095 

(0 

.053) 

0 

.  163 

(0 

.013) 

1 

.335 

(0 

.068) 

0 

.  182 

(0 

.012) 

2 

.686 

(0 

.  196) 

0 

.226 

(0 

.008) 

1 

.778 

(0 

.087) 

0 

.  181 

(0 

.010) 

1 

.983 

(0 

.111) 

0 

.222 

(0 

.010) 

0 

.221 

(0 

.017) 

0 

.162 

(0 

.008) 

1 

.248 

(0 

.051) 

0 

.  173 

(0 

.011) 

0 

.678 

(0 

.035) 

0 

.  135 

(0 

.007) 

2 

.287 

(0 

.  145) 

0 

.359 

(0 

.005) 

2 

.323 

(0 

.110) 

0 

.121 

(0 

.007) 

-596 


64. 


Table  E.27 

1986  IRT  Parameters,  Science,  Grade  11/Age  17 
Physics  Subscale 


FIELD 

I TEM 

NA lOAOl 

S2 

15 

0 

NA 10501 

S2 

22 

0 

NA10601 

S2 

23 

1 

NA10602 

S2 

2A 

0 

NA 10603 

S2 

25 

\ 

NA 1060A 

S2 

26 

Q 

NA06901 

S2 

27 

Q 

NA07A01 

S2 

28 

Q 

NA07A02 

S2 

29 

n 
u 

KA  0720 1 

S2 

0  ^ 

n 
u 

NAO  700 1 

3  3 

Q 

NA 1070 1 

S2 

3  A 

Q 

NA0710 1 

S2 

3  8 

1 

NA 1 OfiO 1 

S2 

T  Q 

n 
u 

NA 1090 1 

S2 

A  0 

Q 

NAl  1001 

S2 

A  1 

Q 

KL 1 1  on  1 

S3 

T  1 

u 

NA2190 1 

S4 

1 A 

u 

NA3  080 1 

SA 

^  7 

Q 

NA29A01 

SA 

19 

0 

NA21601 

SA 

2A 

0 

NA30501 

SA 

26 

0 

NA30601 

SA 

28 

0 

NA32101 

S8 

16 

0 

NA31A01 

S8 

20 

0 

NA21701 

SB 

26 

0 

NA22A01 

S8 

27 

0 

NA31101 

SB 

31 

0 

NA23801 

S9 

18 

u 

NA25001 

S9 

20 

Q 

NA2A801 

S9 

2A 

0 

NA2A802 

S9 

25 

0 

NA2A901 

S9 

28 

0 

NA25501 

S9 

29 

0 

NA26101 

SIO 

16 

0 

NA22201 

SIO 

17 

0 

NA26A01 

SIO 

20 

0 

NA25601 

SIO 

27 

0 

NA26201 

SIO 

32 

0 

NA27301 

SIO 

35 

0 

NA35A01 

Sll 

10 

0 

NA22101 

Sll 

11 

0 

NA37001 

Sll 

16 

1 

NA35901 

Sll 

23 

1 

A  SE  B 


3  A  9 

( 0 

036) 

u 

QIC 

.  0 1  b 

A  02 

( 0 

036 ) 

ft 
u 

0  10 

909 

( 0 

051 ) 

1 

Q  ft  ft 
9  U  U 

A  30 

f  ft 

ft  K  1  > 
U  D  1  ^ 

"2 

279 

^  L'\ 
m  D 

f  ft 

ft  Zi  0  > 

1 

.580 

T  ^  1 

/  ft 

ft  A  7  ^ 

~2 

51 A 

DDv 

^  ft 

ft  'J  > 
U  J  J  ; 

0 

017 

^  ft 

0  A  1 ) 

0 

59 1 

9  Zi  n 

f  ft 

0  A  0 ) 

2 

515 

^  ^  ^ 

^  ft 

032) 

0 

762 

3  68 

( 0 

ft  9  R  > 

0 

2  A  1 

7  5A 

f  ft 

ft  Zi  1  > 

1 

A23 

1 52 

^  ft 

ft  T  R  > 
U  J  0  ; 

2 

019 

/  1  o 

^  ft 

ft  A  ft  \ 

1 

828 

Q  7  1 

^  ft 

ft  Q  ^  ^ 
U  J  0  ; 

1 

7  7  A 

Q  T  9 
9  JZ 

^  ft 

ft  A  ^  ^ 
U  *♦  0  ; 

2 

A  52 

D  O  J 

f  ft 

ft    7  ^ 
U  O  / ; 

2 

503 

681 

^  ft 

ft  Zi  ^  ^ 

-  0 

076 

805 

( 0 

039) 

0 

781 

8  A  8 

( 0 

059 ) 

2 

63  A 

13  A 

( 0 

029 ) 

c 
D 

^99 

ozz 

7  46 

( 0 

0  A  3 ) 

1 
1 

9  9  Q 

A  02 

( 0 

037) 

ft 

u 

803 

900 

( 0 

055) 

Q  R  ft 
O  O  U 

820 

( 0 

055) 

ft 

u 

131 

3  50 

( 0 

03  7) 

Z 

Q  1  Q 

9  1  9 

666 

( 0 

0  A  7 ) 

Z 

Q  ft  ^ 
O  U  0 

5  A  8 

f  ft 

ft  R  ^ 
U  J  o ; 

1 

376 

58A 

( 0 

0A2 ) 

-  ft 

u 

ft  1  9 
U  1  z 

A88 

(0 

OAO) 

-0 

312 

783 

(0 

0A5) 

0 

561 

873 

(0 

031) 

1 

397 

388 

(0 

033) 

0 

721 

810 

(0 

OAO) 

0 

7B2 

320 

(0 

OAO) 

-0 

561 

6A1 

(0 

OAO) 

0 

3A7 

2A3 

(0 

036) 

2 

055 

335 

(0 

095) 

6 

6A1 

838 

(0 

0A9) 

1 

569 

AAB 

(0 

OAB) 

2 

537 

B6A 

(0 

050) 

-0 

513 

378 

(0 

029) 

-0 

907 

033 

(0 

037) 

2 

709 

15A 

(0. 

0A6) 

2 

079 

SE  C  SE 


f  ft 

.  0  A  A  ) 

0 

.267 

( 0 

.017) 

f  ft 

ft  /■  1  ^ 

0 

.  228 

( 0 

.018) 

f  ft 

.  102 ) 

0 

.  129 

( 0 

.  007 ) 

( 0 

.  326) 

0 

3  A  3 

( 0 

.  03A ) 

( 0 

.  09A ) 

0 

.335 

( 0 

.  010 ) 

( 0 

3  A  1 ) 

0 

3A  8 

(0 

.  032) 

( 0 

022) 

0 

195 

(0 

.012) 

( 0 

05A ) 

0 

321 

(0 

.015) 

( 0 

A26  ) 

0 

386 

(0 

OlA  ) 

( 0 

050 ) 

0 

219 

(0 

009) 

( 0 

029 ) 

0 

219 

(0 

012) 

( 0 

0  89) 

0 

26A 

(0 

012) 

( 0 

091 ) 

0 

160 

( 0 

006) 

( 0 

113) 

0 

223 

(0 

Oil) 

( 0 

080 ) 

0 

123 

( 0 

009) 

( 0 

1A2 ) 

0 

17B 

(0 

008) 

( 0 

230 ) 

0 

206 

(0 

010) 

( 0 

023 ) 

0 

217 

(0 

012) 

( 0 

0  A  6 ) 

0 

0 

( 0 

0  ) 

(  0 

203 ) 

0 

20  A 

( 0 

008 ) 

/  1 
^  1 

395) 

0 

202 

( 0 

007 ) 

f  ft 

ft  R  9  ^ 
U  o  Z  ; 

0 

225 

( 0 

012) 

f  ft 

ft  R  ft  ^ 
U  o  U  ; 

0 

205 

( 0 

015) 

f  ft 

OAO) 

0 

176 

( c 

016) 

( 0 

033) 

0 

187 

( 0 

017 ) 

( 0 

30  8) 

0 

220 

( 0 

007 ) 

f  ft 

1  7  % 

0 

285 

( 0 

007 ) 

f  ft 

101) 

0 

183 

( 0 

012) 

f  0 

029 ) 

ft 

u 

1  R  7 

( 0 

018) 

(0 

038) 

0 

193 

(0 

020) 

(0 

0A7) 

0 

226 

(0 

015) 

(0 

058) 

0 

0 

(0 

0  ) 

(0 

067) 

0 

227 

(0 

015) 

(0 

051) 

0 

152 

(0 

013) 

(0 

075) 

0 

209 

(0 

020) 

(0 

031) 

0 

195 

(0 

010) 

(0 

308) 

0 

219 

(0 

013) 

(1 

885) 

0 

1A7 

(0 

007) 

(0 

lOA) 

0 

210 

(0 

Oil) 

(0 

277) 

0 

206 

(0 

Oil) 

(0 

037) 

0 

1B5 

(0 

OlA) 

(0 

072) 

0 

209 

(0 

015) 

(0 

113) 

0 

0 

(0 

0  ) 

(0 

106) 

0 

(0 

005) 

-597 


643 


Table  E.28 

1986  IRT  Parameters,  Science,  Grade  11/Age  17 
Earth  and  Space  Sciences  Subscale 


FIELD       BLOCK  ITEM        A  SE 


N406301 

SI 

21 

0. 

328 

(0 

034) 

N406302 

SI 

22 

0. 

427 

(0. 

036) 

N406303 

SI 

23 

0. 

691 

(0. 

041) 

N40630A 

SI 

24 

0. 

537 

(0. 

039) 

N406601 

SI 

28 

0. 

628 

(0. 

039) 

N410201 

SI 

32 

0. 

674 

(0. 

046) 

N408101 

SI 

38 

0. 

861 

(0. 

041) 

N406401 

S2 

10 

0. 

675 

(0. 

044) 

N406402 

S2 

11 

1. 

042 

(0. 

054) 

N406403 

S2 

12 

0. 

960 

(0. 

077  ) 

N406404 

S2 

13 

1. 

373 

(0. 

089) 

N406405 

S2 

14 

0. 

865 

(0. 

059) 

N406801 

S2 

16 

1. 

084 

(0. 

053) 

N406802 

S2 

17 

1. 

085 

(0. 

055) 

N406803 

S2 

18 

0. 

958 

(0. 

050) 

N406804 

S2 

19 

0 

899 

(0 

043) 

N406805 

S2 

20 

1 

630 

(0 

072) 

N406806 

S2 

21 

0 

454 

(0 

030) 

N40770 1 

S2 

35 

0 

637 

(0 

027) 

N407301 

S2 

36 

0 

326 

(0 

024) 

N407302 

S2 

37 

0 

711 

(0 

048) 

N434801 

S4 

15 

0 

.728 

(0 

039) 

N434301 

S4 

18 

1 

.  192 

(0 

.043) 

N435201 

S5 

6 

0 

.908 

(0 

.055) 

N417601 

S6 

13 

1 

.283 

(0 

.059) 

N418001 

S6 

15 

0 

.711 

(0 

.032) 

N435101 

S6 

16 

0 

.820 

(0 

.034) 

N417801 

S6 

19 

1 

.488 

(0 

.040) 

N417701 

S6 

21 

1 

.855 

(0 

.050) 

N414401 

S6 

23 

0 

.937 

(0 

.030) 

N416801 

S6 

25 

1 

.025 

(0 

.033) 

K417901 

S6 

26 

1 

.050 

(0 

.031) 

N428501 

S7 

23 

1 

.619 

(0 

.074) 

N42g201 

S7 

31 

0 

.267 

(0 

.032) 

N428401 

S7 

34 

0 

.875 

(0 

.050) 

N436801 

Sll 

13 

1 

.  195 

(0 

.044) 

N436802 

Sll 

14 

1 

.455 

(0 

.048) 

N437101 

Sll 

17 

1 

.  174 

(0 

.048) 

N417401 

Sll 

26 

1 

.276 

(0 

.044) 

B  SE  C  SE 


-0. 

658 

(0. 

073) 

0. 

486 

(0. 

014) 

0. 

638 

(0. 

063) 

0  . 

458 

(0. 

012) 

0. 

971 

(0. 

070  ) 

0. 

452 

(0. 

010) 

0. 

618 

(0. 

056) 

0  . 

395 

(0. 

012) 

0. 

129 

(0. 

026) 

0  . 

257 

(0. 

013) 

2. 

454 

(0. 

182) 

0. 

250 

(0. 

oil) 

2. 

224 

(0. 

121) 

0  . 

165 

(0. 

010) 

0. 

553 

(0. 

052) 

0  . 

532 

(0. 

oil) 

0. 

598 

(0. 

051) 

0  . 

439 

(0. 

oil) 

-0. 

387 

(0. 

051) 

0. 

533 

(0. 

015) 

0. 

137 

(0. 

047  ) 

0  . 

464 

(0. 

013) 

0. 

024 

(0. 

041) 

0. 

486 

(0. 

015) 

-0. 

834 

(0. 

062) 

0. 

420 

(0. 

020) 

1. 

837 

(0. 

122) 

0  . 

523 

(0. 

007  ) 

-0. 

404 

(0. 

036) 

•  0. 

321 

(0. 

015) 

-0 

681 

(0 

044) 

0 

339 

(0. 

017) 

1 

526 

(0 

119) 

0 

601 

(0 

007  ) 

0 

4  92 

(0 

042) 

0 

376 

(0 

012) 

1 

161 

(0 

057) 

0 

167 

(0 

009) 

1 

568 

(0 

120) 

0 

239 

(0 

009) 

2 

258 

(0 

165) 

0 

426 

(0 

008) 

1 

.442 

(0 

090) 

0 

314 

(0 

012) 

1 

.  193 

(0 

.062) 

0 

.  178 

(0 

.012) 

0 

.356 

(0 

.036) 

0 

.311 

(0 

.010) 

0 

.247 

(0 

.033) 

0 

.278 

(0 

.010) 

-0 

.009 

(0 

.022) 

0 

.199 

(0 

.012) 

0 

.114 

(0 

.023) 

0 

.  175 

(0 

.011) 

0 

.987 

(0 

.047) 

0 

.229 

(0 

.007) 

0 

.926 

(C 

.052) 

0 

.264 

(0 

.007) 

1 

.030 

(0 

.043) 

0 

.248 

(0 

.006) 

1 

.309 

(0 

.057) 

0 

.277 

(0 

.007) 

1 

.766 

(0 

.067) 

0 

.  193 

(0 

.006) 

0 

.690 

(0 

.061) 

0 

.240 

(0 

.015) 

1 

.994 

(0 

.243) 

0 

.271 

(0 

.014) 

2 

.428 

(0 

.  159) 

0 

.236 

(0 

.009) 

1 

.167 

(0 

.063) 

0 

.199 

(0 

.012) 

1 

.781 

(0 

.  094) 

0 

.234 

(0 

.009) 

1 

.905 

(0 

.  107) 

0 

.312 

(0 

.010) 

1 

.821 

(0 

.092) 

0 

.281 

(0 

.007) 
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Tablt  E.29 

1986  IRT  Ptrameteri,  U.S.  History,  Grad«  ll/Ag«  17 


FIELD 

BLOCK 

ITEM 

-^000101 

HI 

13 

H000201 

HI 

14 

H000301 

HI 

15 

HOOOAOl 

HI 

16 

H000501 

HI 

17 

H000601 

HI 

16 

H000701 

HI 

19 

Hoooeoi 

HI 

20 

H000901 

HI 

21 

HOO-lOOl 

HI 

22 

HOOllOl 

HI 

23 

H001201 

■  HI 

24 

H001202 

HI 

25 

H001203 

HI 

26 

H001204 

HI 

27 

H001205 

HI 

28 

H001301 

HI 

29 

H001401 

HI 

30 

H001501 

HI 

31 

B001601 

HI 

32 

H001701 

HI 

33 

H001801 

HI 

34 

Hooigol 

HI 

35 

H002001 

HI 

36 

H002101 

HI 

37 

H002201 

HI 

38 

H002301 

HI 

39 

H002401 

HI 

40 

H002402 

HI 

41 

H002403 

HI 

42 

H002404 

HI 

43 

H002S01 

HI 

44 

H002601 

HI 

45 

H002701 

HI 

46 

H002e01 

HI 

47 

H002901 

HI 

48 

H003001 

H2 

13 

H003101 

H2 

14 

H003201 

H2 

15 

H003301 

H2 

16 

H003401 

H2 

17 

H003501 

H2 

18 

H003601 

H2 

19 

HC0240S 

H2 

20 

H002406 

H2 

21 

H002407 

H2 

22 

H002406 

H2 

23 

H003701 

H2 

24 

H003601 

H2 

25 

H003901 

H2 

26 

H004001 

H2 

27 

H004 101 

H2 

28 

H004201 

H2 

29 

H004301 

H2 

30 

H004401 

H2 

31 

H004S01 

H2 

32 

H004S02 

H2 

33 

H004601 

H2 

34 

H004701 

H2 

35 

H004801 

H2 

36 

H004901 

H2 

37 

HOOSuOl 

H2 

38 

HOOSlOl 

H2 

39 

H00S102 

H2 

40 

H005103 

H2 

41 

H00S201 

H2 

42 

H005301 

H2 

43 

H005401 

H2 

44 

H005501 

H2 

45 

H00S601 

H2 

46 

H005701 

H2 

47 

H005801 

H2 

48 

H00S901 

H3 

13 

A  SE  B 


1 

.  159 

(0 

.086) 

-0 

.  128 

1 

.291 

(0 

.  153) 

1 

.224 

0 

.601 

(0 

.052) 

-1 

.220 

0 

.407 

(0 

.054) 

-0 

.381 

1 

.  181 

(0 

.  103) 

0 

.  147 

0 

.364 

(0 

.044) 

-0 

.447 

1 

.556 

(0 

.  159) 

1 

.989 

0 

.900 

(0 

.084  ) 

-2 

.362 

1 

.  167 

(0 

.  106) 

0 

.788 

0 

.751 

(0 

.069) 

0 

.056 

0 

.625 

(0 

.061) 

-0 

147 

1 

.813 

(0 

.  123) 

-0 

397 

1 

.208 

(0 

.095) 

-1 

239 

0 

.673 

(0 

.059) 

-0 

244 

1 

.447 

(0 

.  112) 

-0 

479 

1 

.401 

(0 

.  120) 

-1 

517 

1 

.059 

(0 

.091) 

-0 

445 

0 

.385 

(0 

.041) 

-1 

202 

0 

.728 

(0 

.057) 

-0 

563 

0 

.601 

(0 

054  ) 

-1 

495 

1 

.  106 

(0 

106) 

0 

486 

0 

882 

(0 

156) 

1 

664 

0 

383 

(0 

058) 

0 

658 

0 

.529 

(0 

062) 

0 

203 

1 

064 

(0 

110) 

0 

704 

0 

602 

(0 

060) 

-0 

798 

1 

.190 

(0 

147) 

1 

839 

0 

444 

(0 

088) 

1 

622 

0 

828 

(0 

067) 

-1 

466 

0 

911 

(0 

107) 

1 

551 

0 

924 

(0 

145) 

1 

741 

0 

651 

(0 

057) 

-0 

922 

0 

878 

(0 

104) 

0 

706 

0 

791 

(0 

101) 

1 

220 

1 

432 

(0 

168) 

1 

962 

0 

848 

(0 

075) 

-0 

754 

1 

209 

(0 

097) 

-0 

740 

0 

725 

(0 

071) 

0 

025 

0 

680 

(0 

101) 

1 

428 

0 

758 

(0 

082) 

0 

268 

1 

337 

(0 

191) 

1 

990 

0 

973 

(0 

137) 

1 

199 

0 

966 

(0 

104) 

0 

474 

1 

772 

(0 

170) 

0 

938 

?. 

475 

(0 

237  ) 

0 

665 

1 

785 

(0 

179) 

0 

942 

1 

175 

(0 

129) 

0 

554 

1 

560 

(0 

178) 

1 

426 

0 

952 

(0 

103) 

1 

109 

0 

836 

(0 

075) 

-0 

086 

0 

822 

(0 

097) 

0 

516 

0 

989 

(0 

122) 

0 

377 

0 

693 

(0 

319) 

4. 

222 

0 

380 

(0 

098) 

2. 

340 

0 

862 

(0 

094  ) 

-0. 

113 

0 

327 

(0 

041) 

-2 

027 

0 

593 

(0 

055) 

-0. 

575 

0 

484 

(0 

050) 

0. 

093 

0 

588 

(0 

053) 

-1. 

938 

1 

131 

(0 

168) 

1. 

628 

0 

464 

(0 

046) 

-0. 

765 

1 

437 

(0. 

113) 

-0. 

612 

1 

468 

(0 

149) 

1. 

067 

1 

357 

(0. 

097) 

-0. 

506 

1 

250 

(0 

094  ) 

-0. 

313 

0 

792 

(0. 

064  ) 

0. 

297 

0. 

390 

(0. 

068) 

0. 

923 

0 

921 

(0. 

074  ) 

-0. 

325 

0 

640 

(0. 

095) 

1. 

417 

1 

041 

(0. 

097) 

0. 

284 

0. 

822 

(0. 

105) 

1. 

162 

0. 

676 

(0. 

060) 

-1. 

415 

1. 

055 

(0. 

090) 

0. 

171 

-599- 


SE 

c 

SE 

0 

.  067  ) 

0 

.  186 

(0 

.029 

0 

.  247  ) 

0 

.235 

(0 

.017 

0 

.  127  ) 

0 

.216 

(0 

.054 

0 

.  122) 

0 

.301 

(0 

.064 

0 

.  089 ) 

0 

.243 

(0 

.  029 

0 

.  105 ) 

0 

.231 

(0 

.  057 

0 

.339 ) 

0 

.  188 

(0 

.010 

0 

.  248 ) 

0 

.213 

(0 

.  054 

0 

.  134  ) 

0 

.  154 

(0 

.  018 

0 

.  082 ) 

0 

.  196 

(0 

.  038 

0 

.  085 ) 

0 

.202 

(0 

.046 

0 

.  066 ) 

0 

.  153 

(0 

.024 

0 

.  127  ) 

0 

.223 

( 0 

.  046 

0 

.  076 ) 

0 

.  198 

(0 

.041 

0 

.  077  ) 

0 

.258 

(0 

.033 

0 

.  177  ) 

0 

.230 

(0 

.  049 

0 

.  087  ) 

0 

.291 

(0 

.041 

0 

.  149 ) 

0 

.225 

(0 

.  056 

0 

.  078) 

0 

.  179 

(0 

.042 

0 

.  154  ) 

0 

.236 

(0 

.059 

0 

.  118) 

0 

.222 

(0 

.026 

0 

.  386) 

0 

322 

(0 

.022 

0 

.  160 ) 

0 

234 

(0 

.055 

0 

.  103 ) 

0 

237 

(0 

048 

0 

144  ) 

0 

233 

(0 

.024 

0 

.117) 

0 

264 

(0 

.059 

0 

.  326 ) 

0 

172 

(0 

.012 

0 

369 ) 

0 

260 

(0 

.045 

0 

139 ) 

0 

213 

(0 

051 

0 

236) 

0 

110 

(0 

016 

0 

357 ) 

0 

197 

(0 

013 

0 

108 ) 

0 

215 

(0 

052 

0 

156 ) 

0 

270 

(0 

030 

0 

213 ) 

0 

187 

( 0 

025 

0 

367  ) 

0 

261 

( 0 

012 

0 

102 ) 

0 

252 

( 0 

051 

0 

092 ) 

0 

248 

(0 

042 

0 

087 ) 

0 

226 

(0 

041 

0 

264  ) 

0 

205 

(0 

028 

0 

105 ) 

0 

243 

( 0 

039 

0 

437  ) 

0 

342 

( 0 

013 

0 

254  ) 

0 

294 

( 0 

023 

0 

125 ) 

0 

281 

( 0 

030 

n 

u 

0 

208 

( 0 

014 

u 

9 1 1  ^ 

Z  1 1  J 

0 

334 

( 0 

015 

u 

Z  1  9  J 

0 

275 

( 0 

015 

0 

146) 

0 

315 

(0 

026 

u 

Q  n  A  ^ 
o  U  *t  J 

0 

393 

( 0 

014 

u 

1  /  / ; 

0 

143 

( 0 

020 

u 

n  7  Q  ^ 

0 

211 

( 0 

038 

U 

LOO  J 

0 

299 

( 0 

034 

u 

±H  o  ) 

0 

399 

( 0 

033 

z 

Id/  } 

0 

190 

( 0 

010 

U 

a^Q  \ 
onii  ) 

0 

354 

( 0 

04  2 

0 

104  ) 

0 

357 

( 0 

04  3 

0 

265 ) 

0 

253 

( 0 

064 

0 

093  ) 

0 

216 

( 0 

04  9 

0 

081 ) 

0 

179 

( 0 

043 

0 

186 ) 

0 

205 

( 0 

052 

u 

O  0  1  J 

0 

209 

(0 

016 

0 

107) 

0 

213 

(0 

052 

0 

084) 

0. 

242 

(0 

036 

n 

205) 

0. 

163 

(0 

015 

070) 

0. 

160 

(0 

032 

0 

068) 

0. 

184 

(0 

032 

0 

073) 

0. 

131 

(0. 

027 

0 

217) 

0. 

268 

(0. 

055 

0 

073) 

0. 

188 

(0. 

037 

0 

257) 

0. 

190 

(0. 

029 

0. 

097) 

0. 

220 

(0. 

029 

0 

210) 

0. 

191 

(0. 

025 

0. 

145) 

0. 

226 

(0. 

055 

0. 

082) 

0. 

217 

(0. 

028: 

ERIC 


645 


Table  E.29 
(continued) 


FIELD      BLOCK  ITEM 


H006001 

H006101 

H006201 

H006?01 

H006A01 

H006501 

H006601 

H006701 

H006801 

H006901 

H007001 

H007101 

H007102 

H007103 

H007201 

H007301 

H007401 

H007501 

H007601 

H007701 

H007801 

H007901 

H008001 

HOOBlOl 

H008201 

H008301 

H008302 

H008303 

H008304 

H008305 

H008401 

H008501 

H008601 

H008701 

H008801 

H008901 

H009001 

H009101 

H005004 

H005005 

H005006 

H005007 

H005008 

H005009 

H005010 

H009201 

H009301 

H009401 

H009501 

H009601 

H009701 

H009801 

H009901 

HOlOOOl 

HOIOIOI 

H010201 

H010301 

H010401 

H010501 

H010601 

H010701 

H010801 

H010901 

HOllOOl 

HOlllOl 

H011201 

H011301 

H011401 


H3 
H3 
H3 
H3 
H3 
H3 
H3 
H3 
H3 
H3 
H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 

H3 
H3 

HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 
HA 


lA 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

3A 

35 

36 

37 

38 

39 

AO 

Al 

A2 

A3 

AA 

A5 

A6 

A7 

13 

lA 

15 

16 

17 

18 

19 

20 

21 

22 

23 

2A 

25 

26 

27 

28 

29 

30 

31 

32 

33 

3A 

35 

36 

37 

38 

39 

AO 

Al 

A2 

A3 

AA 

A5 

A6 


A  SE 

025  (0.083) 
115  (0.086) 
6A7  (O.lOA) 
109  (0.101) 
9A2  (0.119) 
621  (0.168) 

0.  636  (0.08A) 
1.035  (0.179) 

1.  A18  (0.  115) 
0.2A6  (0.05A) 
1.173  (0.092) 
0.859  (0.132) 
1.133  (0.090) 
0.8AA  (0.078) 
0.698  (0.118) 
0.859  (0.112) 

(0. 13A) 
(0.087) 
1.189  (0.152) 
1.755  (0.1A2) 
0.759  (0.070) 
0.995  (0.232) 
0.872  (0.139) 
633  (0.070) 


0.77A 
0.881 


535  (0.088) 
092  (0.102) 
801  (0.073) 
688  (0.08A) 
836  (0.072) 
0.62A  (0.111) 
1.098  (0.099) 
0.52A  (0.055) 
0.A26  (0.060) 
0.507  (0.09A) 
0.698  (0.068) 
1.2A9  (0.108) 
0.981  (0.088) 
0.865  (0.070) 
1.087  (0.077) 
0.899  (0.067) 
1.082  (0.085) 
1.718  (0.15A) 
1.300  (0.078) 
0.799  (0.059) 
1.283  (0.101) 
0.A51  (0.050) 
0.80A  (0.113) 
0 . 6A6   (0  .068) 

0.  593  (0.081) 

1.  A23  (0.1A3) 
1.207  (0.095) 
1.162  (0.087) 
0.99A  (0.07A) 
1.06A  (0.088) 
1.090  (0.09A) 
0.560  (0.075) 
0.82A  (0.06A) 
0.925  (0.082) 
0.8A7  (0.066) 
l.lOA  (0.117) 
0.88A  (0.115) 
0.279  (O.OAl) 
0.863  (0.079) 
1.028  (0.081) 
0.80A  (0.081) 
0.610  (0.062) 
1.0A6  (0.078) 
0.505  (0.087) 


B 

-0. A39 
-1.351 
1.613 
,159 
.637 
.572 
.833 
,  193 
.1A9 
.231 
,187 
1.A8A 
0.106 
-0.36A 
1.025 
1.557 
.607 
.  A35 
.138 
.281 
.255 
2.356 
1.693 
0.505 
1.599 
1.203 
0.A21 
0.285 
-0.A38 
1.66A 
.091 
,574 
.676 
.246 
,021 
0.355 
0.825 
-1.4AA 
-0 .003 
-0.402 
-0.015 
1.315 
-0.289 
0.028 

0.  709 
-0.125 

1.  A33 
-0.306 

0.737 
1.318 
-0.533 
-0.797 
-1.363 
-1.058 
0.415 
0.372 
-0.720 
0.839 
-0.8A9 
0.9A5 
1.474 
0.631 
0.373 
-0.321 

0.  827 
-1. 123 
-0.460 

1.  173 


SE 

(0.078) 
(0.132) 
(0.308) 
(0.085) 
(0.267) 
(0.768) 
(0.165) 
(0.489) 
(0.074) 
(0.725) 
(0.077) 
(0  .301) 
(0.074) 
(0.088) 
(0.248) 
(0.259) 
(0.347) 
(0.105) 
(0 .242) 
(0.075) 
(0.084) 
(0  .702) 
(0.344) 
(0.111) 
(0  .303) 
(0  .166) 
(0  .091) 
(0.119) 
(0.083) 
(0  .350) 
(0.087) 
(0  .280) 
(0.149) 
(0.286) 
(0  .086) 
(0.099) 
(0.123) 
(0  .139) 
(0.061) 
(0.069) 
(0.070) 
(0.240) 
(0.052) 
(0.060) 
(0.112) 
(0.089) 
(0.263) 
(0.099) 
(0.164) 
(0.232) 
(0.080) 
(0.089) 
(0.125) 
(0.116) 
(0.097) 
(0.133) 
(0.085) 
(0.120) 
(0.094) 
(0.173) 
(0.256) 
(0.133) 
(0.094) 
(0.072) 
(0.  133) 
(0.144) 
(0.072) 
(0.259) 


0.229 
0  .208 
0.191 
0  .289 
0.120 
0.335 
0.217 
.  167 
.268 
.246 
.  167 
.238 
.189 
.275 
.366 
.  130 
.275 
.206 
0.272 
0  .270 
0.176 
0  .338 
0.211 
0.183 
0  .  178 
0.092 
0  .  151 
0.278 
0  .225 
0.252 
0.246 
0  .233 
0.223 
0.299 
0  .207 
0.230 
0.143 
0.235 
0.116 
0.149 
0.175 
0.300 
0.085 
0.109 
0.113 
0.212 
0.223 
0.254 
0.242 
0.187 
0.251 
0.220 
0.  194 
0.269 
0.190 
0.270 
0.198 
0.125 
0.211 
0.232 
0.183 
0.235 
0.195 
0  .201 
0.154 
0.284 
0.192 
0.250 


SE 

(0.039) 
(0.047) 
(0.027) 
(0.036) 
(0.015) 
(0.023) 
(0.036) 
(0  .013) 
(0.029) 
(0.037) 
(0.024) 
(0.022) 
(0.026) 
(0.044) 
(0  .035) 
(0.018) 
(0.024) 
(0.030) 
(0.019) 
(0.028) 
(0.033) 
(0.015) 
(0.020) 
(0.037) 
(0.033) 
(0.014) 
(0.029) 
(0.044) 
(0.044) 
(0.029) 
(0.031) 
(0.059) 
(0.048) 
(0.043) 
(0.041) 
(0.024) 
(0.020) 
(0.053) 
(0.025) 
(0.036) 
(0.029) 
(0.013) 
(0.021) 
(0.028) 
(0.016) 
(0. 050) 
(0.023) 
(0.053) 
(0.042) 
(0.013) 
(0.038) 
(0.040) 
(0.046) 
(0.050) 
(0.024) 
(0.050) 
(0.043) 
(0.020) 
(0.046) 
(0.021) 
(0.020) 
(0.048) 
(0  .030) 
(0.036) 
(0.026) 
(0.065) 
(0.036) 
(0.045) 
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Table  E.30 

1986  IRT  Parameters.  Literature,  Grade  11/Aso  17 


FIELD 

BLOCK 

ITEM 

LOOOlOl 

LI 

19 

L000201 

LI 

20 

L000301 

LI 

21 

LOOOAOl 

LI 

22 

L000501 

LI 

23 

L000601 

LI 

2A 

L000701 

LI 

25 

L000801 

LI 

26 

L000901 

LI 

27 

LOOlOOl 

LI 

28 

LOOllOl 

LI 

29 

L001201 

LI 

30 

L001301 

LI 

31 

LOOlAOl 

LI 

32 

L001501 

LI 

33 

L001601 

LI 

34 

L001701 

LI 

35 

L001801 

LI 

36 

L001901 

LI 

37 

L002001 

LI 

38 

L002101 

LI 

39 

L002201 

LI 

AO 

L002301 

LI 

Al 

L002A01 

LI 

A2 

L002501 

LI 

A3 

L002601 

LI 

AA 

L002701 

LI 

A5 

L002801 

LI 

A6 

L002901 

LI 

A7 

L003001 

LI 

A8 

L003101 

L2 

19 

L003201 

L2 

20 

L003301 

L2 

21 

L003A01 

L2 

22 

L003501 

L2 

23 

L003601 

L2 

2A 

L003701 

L2 

25 

L003801 

L2 

26 

L003901 

L2 

27 

L004001 

L2 

28 

L004101 

L2 

29 

L004201 

L2 

30 

L00A301 

L2 

31 

LOOAAOl 

L2 

32 

LOOASOl 

L2 

33 

L004601 

L2 

3A 

L00A701 

L2 

35 

L00A801 

L2 

36 

L00A901 

L2 

37 

L005001 

L2 

38 

L005101 

L2 

39 

L005201 

L2 

AO 

L005301 

L2 

Al 

L005401 

L2 

A2 

L005501 

L2 

A3 

L005601 

L2 

AA 

L005701 

L2 

A5 

L005801 

L2 

A6 

L005901 

L2 

A7 

L006001 

L2 

A8 

L006101 

L2 

A9 

L006201 

L3 

19 

L006301 

L3 

20 

L006401 

L3 

21 

L006501 

L3 

22 

L006601 

L3 

23 

L006701 

L3 

2A 

L006801 

L3 

25 

L006901 

L3 

26 

L007001 

L3 

27 

L007101 

L3 

L007201 

L3 

r.d 

L007301 

L3 

30 

ERIC 


A  SE  B  SE  C  SE 


0. 

763 

(0. 

095) 

0  . 

A23 

(0 . 

132) 

0 . 

320 

( 0 . 

038 ) 

0. 

632 

(0  . 

06A ) 

0  . 

165 

(0 . 

087 ) 

0 . 

185 

( 0 . 

040 ) 

0. 

A71 

(0  . 

117) 

2 . 

701 

(0 . 

714  ) 

0 . 

262 

( 0 . 

029 ) 

0. 

765 

(0  . 

065) 

-0 . 

561 

(0 . 

084  ) 

0 . 

212 

( 0 . 

044  ) 

1. 

063 

(0  . 

12A ) 

1 . 

lOA 

( 0 . 

202) 

0 . 

209 

( 0 , 

020 ) 

0 . 

373 

(0, 

OAl) 

-1 . 

195 

( 0 . 

152) 

0 . 

223 

( 0 . 

057 ) 

0 

908 

(0  . 

083) 

0 . 

216 

( 0 . 

085) 

0 . 

205 

(0 , 

031 ) 

0 

755 

(0. 

087  ) 

0. 

872 

(0 

151) 

o! 

192 

(0 

028) 

0 

512 

(0, 

09A) 

1. 

614 

(0 

344  ) 

0. 

256 

(0 

037 ) 

0 

965 

(0. 

099) 

0. 

963 

(0 

153) 

0 . 

145 

(0 

021 ) 

0 

8A7 

(0  . 

115) 

1 . 

396 

(0 

253 ) 

0 . 

221 

(0 

022) 

0 

652 

(0. 

068) 

0. 

263 

(0 

093 ) 

0 . 

205 

(0 

038) 

0 

8A1 

(0, 

090) 

0. 

A98 

(0 

117  ) 

0 . 

217 

(0 

032) 

1 

36A 

(0  . 

113) 

0. 

09A 

(0 . 

080 ) 

0 . 

228 

(0 

025 ) 

0 

898 

(0 

15A  ) 

1. 

369 

(0 

328) 

0 . 

396 

(0 

024 ) 

0 

763 

(0. 

062) 

-1 

398 

(0 

134  ) 

0 . 

217 

(0 

053) 

0 

3A7 

(0 

055) 

0 

692 

(0 

163) 

0 . 

267 

( 0 

053) 

1 

152 

(0 

135) 

1 

212 

(0 

225) 

0 . 

210 

( 0 

018 ) 

0 

353 

(0 

0A8) 

0 

062 

(0 

102) 

0 

239 

( 0 

056 ) 

1 

776 

(0 

167) 

2 

078 

(0 

361) 

0 

131 

( 0 

008) 

0 

985 

(0 

07A) 

-0 

791 

( 0 

086) 

0 

193 

( 0 

040 ) 

0 

088 

(0 

023) 

10 

790 

(2 

792) 

0 

091 

( 0 

020 ) 

0 

5A0 

(0 

083) 

1 

528 

( 0 

273 ) 

0 

179 

( 0 

032 ) 

0 

902 

(0 

090) 

-0 

021 

( 0 

091 ) 

0 

268 

( 0 

039 ) 

0 

765 

(0 

06A) 

-1 

655 

( 0 

159) 

0 

225 

( 0 

056) 

1 

085 

(0 

151) 

1 

373 

( 0 

286) 

0 

278 

( 0 

019) 

0 

901 

(0 

092) 

0 

229 

(0 

100 ) 

0 

265 

(0 

034 ) 

0 

561 

(0 

079) 

1 

362 

(0 

230) 

0 

166 

(0 

031) 

0 

6AA 

(0 

073) 

0 

116 

(0 

099) 

0 

240 

(0 

045) 

0 

9A2 

(0 

079) 

-0 

532 

(0 

0^3) 

0 

227 

(0 

042) 

0 

955 

(0 

080) 

-1 

271 

(0 

132) 

0 

248 

(0 

054 ) 

1 

021 

(0 

150) 

1 

366 

(0 

291) 

0 

276 

(0 

020) 

0 

552 

(0 

088) 

1 

2A5 

(0 

247) 

0 

219 

(0 

038) 

0 

383 

(0 

OAl) 

-1 

919 

(0 

219) 

0 

216 

(0 

056) 

0 

931 

(0 

1A7) 

1 

988 

(0 

397) 

0 

152 

(0 

015) 

0 

531 

(0 

05A) 

-0 

783 

(0 

115) 

0 

243 

(0 

057 ) 

1 

202 

(0 

16A) 

1 

007 

(0 

242) 

0 

368 

(0 

021) 

0 

A20 

(0 

062) 

1 

120 

(0 

202) 

0 

186 

(0 

042) 

0 

392 

(0 

052) 

0 

267 

(0 

106) 

0 

224 

(0 

051 ) 

0 

231 

(0 

062) 

4 

A97 

(1 

220) 

0 

269 

(0 

034  ) 

1 

A51 

(0 

112) 

-1 

162 

(0 

129) 

0 

203 

(0 

041) 

0 

A75 

(0 

093) 

2 

002 

(0 

432) 

0 

218 

(0 

034  ) 

0 

869 

(0 

lAl) 

0 

957 

(0 

248) 

0 

429 

(0 

029) 

0 

5AA 

(0 

068) 

1 

113 

(0 

176) 

0 

155 

(0 

032) 

0 

83A 

(0 

102) 

0 

816 

(0 

162) 

0 

249 

(0 

029) 

1 

201 

(0 

092) 

-0 

817 

(0 

092) 

0 

211 

(0 

040) 

0 

618 

(0 

066) 

0 

283 

(0 

095) 

0 

194 

(0 

040) 

1 

09A 

(0 

172) 

2 

A12 

(0 

504) 

0 

106 

(0 

010) 

0 

773 

(0 

07A) 

-0 

202 

(0 

084) 

0 

237 

(0 

043) 

1 

531 

(0 

228) 

2 

867 

(0 

699) 

0 

142 

(0 

.008) 

1 

101 

(0 

09A) 

-1 

883 

(0 

198) 

0 

217 

(0 

.053) 

0 

.AA9 

(0 

181) 

5 

616 

(2 

363) 

0 

177 

(0 

.011) 

0 

.982 

(0 

108) 

1 

297 

(0 

198) 

0 

118 

(0 

.017) 

1 

.157 

(0 

.128) 

0 

8A2 

(0 

.169) 

0 

252 

(0 

.021) 

1 

.098 

(0 

086) 

-0 

751 

(0 

089) 

0 

226 

(0 

.041) 

1 

.217 

(0 

.115) 

0 

901 

(0 

.149) 

0 

134 

(0 

.01?) 

0 

.671 

(0 

.059) 

-0 

.A71 

(0 

.082) 

0 

196 

(0 

.044) 

0 

.  AAO 

(0 

.113) 

2 

.65A 

(0 

.724) 

0 

.281 

(0 

.034) 

0 

.6A6 

(0 

.067  ) 

-0 

.22A 

(0 

.090) 

0 

.240 

(0 

.048) 

0 

.699 

(0 

.071) 

-0 

.03A 

(0 

.087) 

0 

.217 

(0 

.043) 

1 

.196 

(0 

.  116) 

0 

.788 

(0 

.  143) 

0 

.  149 

(0 

.020) 

1 

.A23 

(0 

.103) 

-1 

.001 

(0 

.  105) 

0 

.  176 

(0 

.036) 

0 

.35A 

(0 

.057) 

1 

.  38A 

(0 

.253) 

0 

.  192 

(0 

.044) 

0 

.2A5 

(0 

.071) 

5 

.  454 

(1 

.598) 

0 

.  192 

(0 

.029) 

0 

.605 

(0 

.  05A) 

-1 

.  097 

(0 

.120) 

0 

.219 

(0 

.054) 

0 

.7AA 

(0 

.  131) 

1 

.974 

(0 

.413) 

0 

.  183 

(0 

.021) 

0 

.5A8 

(0 

.050) 

-2 

.051 

(0 

.201) 

0 

.214 

(0 

.055) 

0 

.771 

(0 

.  103) 

0 

.891 

(0 

.  183) 

0 

.281 

(0 

.031) 

0 

.65A 

(0 

.092) 

0 

.787 

(0 

.174) 

0 

.284 

(0 

.038) 

0 

.  A3A 

(0 

.0A9) 

-0 

.042 

(0 

.086) 

0 

.203 

(0 

.049) 

0 

.7A5 

(0 

.121) 

1 

.303 

(0 

.281) 

0 

.325 

(0 

.029) 

0 

.855 

(0 

.079) 

-0 

.567 

(0 

.095) 

0 

.282 

(0 

.048) 

1 

.0A6 

(0 

.  119) 

1 

.221 

(0 

.207) 

0 

.  176 

(0 

.018) 
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Table  £.30 
(continued) 


FIELD  BLOCK 

ITEM 

SE 

L007401 

L3 

31 

0. 

550 

C  0  . 

ODD  J 

L007501 

L3 

32 

0. 

852 

/  f\ 

(  0  . 

IZA  J 

L007601 

L3 

33 

0. 

35A 

/  f\ 

(  0  . 

UDd  ; 

LOO  7701 

L3 

3A 

0. 

733 

C  0 . 

n  RH  ^ 
MOsJ  ) 

L007801 

L3 

35 

0. 

703 

C  0  . 

0  /  o  ; 

L007901 

L3 

36 

1. 

183 

/  f\ 

(  0  . 

1  T  o  \ 
1/  Zj 

L008001 

L3 

37 

0. 

385 

(0 . 

0^5 ) 

L008101 

L3 

38 

0. 

AOl 

0  A  D  ; 

L008201 

L3 

39 

0. 

788 

(0 . 

069 ) 

L008301 

L3 

AO 

1. 

613 

1  Q'a  \ 
IsO  J 

LOOS^Ol 

L3 

Al 

0. 

517 

( 0 . 

0 oA  J 

L008501 

L3 

A2 

0. 

887 

c  0 . 

1  1  ^  ^ 

L008601 

L3 

A3 

0. 

986 

c  0 . 

yj  /  D  J 

L008701 

L3 

AA 

0. 

312 

( 0 . 

0  A  5 ) 

L008801 

L3 

A5 

0. 

938 

( 0 . 

086  ) 

L008901 

L3 

A6 

0. 

A96 

( 0 . 

065  ) 

L009001 

L3 

A7 

0. 

889 

( 0 . 

065) 

L009101 

L3 

A8 

0. 

7A7 

(0  . 

069 ) 

L009201 

LA 

19 

0. 

591 

(0 . 

051 ) 

L009301 

LA 

20 

0 

238 

( 0 

03  A ) 

L009A01 

LA 

21 

1. 

06A 

(0 

310) 

L009501 

LA 

22 

0 

360 

( 0 

0  A  8 ) 

L009601 

LA 

23 

0 

596 

(0 

19A ) 

L009701 

LA 

2A 

1 

156 

( 0 

0  92 ) 

L009801 

LA 

25 

0 

A55 

( 0 

076) 

L009901 

LA 

26 

0 

907 

( 0 

093) 

LOlOOOl 

LA 

27 

0 

.399 

( 0 

.  0  A  5 ) 

LOlO 101 

LA 

28 

0 

.587 

(  0 

1  ^ 

L010201 

LA 

29 

0 

.778 

( 0 

.061) 

LO 10301 

LA 

30 

0 

.  lAl 

C  0 

n  1 9  ^ 
.  UOZ  ) 

LOlOAOl 

LA 

31 

0 

.693 

K  0 

.  yJDO  ) 

LO IODO 1 

LA 

32 

0 

.  736 

f  n 

.  087  ) 

LO lObOl 

T  A 

33 

0 

.503 

f  n 

.  050 ) 

LO 10  701 

LA 

3A 

0 

.97A 

f  n 

.073  ) 

T  n  1  n  ft  n  1 

LA 

35 

0 

.920 

(0 

.  1A6 ) 

L010901 

LA 

36 

0 

.981 

(0 

.266) 

LOllOOl 

LA 

37 

0 

.  76A 

(0 

.151) 

LOlllOl 

LA 

38 

0 

.  A7A 

(0 

.0A9> 

L011201 

LA 

39 

0 

.509 

(0 

.051) 

L011301 

LA 

AO 

1 

.  138 

(0 

.2A0) 

L011401 

LA 

Al 

0 

.686 

(0 

.  1A8) 

L011501 

LA 

A2 

1 

.OAO 

(0 

.089) 

L011601 

LA 

A3 

0 

.93A 

(0 

.091) 

L011701 

LA 

AA 

0 

.959 

(0 

.099) 

L011801 

LA 

A5 

1 

.20A 

(0 

.  176) 

L011901 

LA 

A6 

1 

.733 

(0 

.  170) 

L012001 

LA 

A7 

0 

.  809 

(0 

.069) 

L012101 

LA 

A8 

0 

.  9AA 

(0 

.068) 

B  SE  C  SE 


-0.077 

(0.080) 

0. 

193 

(0. 

044  ) 

1.585 

(0.296) 

0. 

203 

(0. 

021 ) 

0.867 

(0.182) 

0  . 

226 

(0. 

052) 

-1.565 

(0.1A8) 

0  . 

216 

(0. 

054  ) 

0.853 

(0  .lAO) 

0  . 

164 

(0. 

029) 

2.355 

(0.A72) 

0. 

144 

(0. 

010) 

O.IAO 

(0.086) 

0. 

189 

(0. 

047) 

-O.A02 

(0.097) 

0  . 

219 

(0. 

054  ) 

-2.063 

(0.200) 

0. 

225 

(0. 

057 ) 

2.335 

(0.A77) 

0  . 

176 

(0. 

009) 

1.215 

(0.2A7) 

0. 

252 

(0. 

OAO ) 

0.463 

(0.1A9) 

0. 

391 

(0. 

035) 

-0.589 

(0.077) 

0  . 

193 

(0. 

038) 

0.361 

(0 .110) 

0  . 

227 

(0. 

051 ) 

-0.151 

(0.082) 

0. 

253 

(0. 

039) 

0.588 

(0.13A) 

0. 

215 

(0. 

045) 

-0.575 

(0.072) 

0. 

155 

(0. 

036) 

-0.032 

(0.078) 

0. 

202 

(0. 

039) 

-1.240 

(0.128) 

0. 

215 

(0. 

053) 

-0. 936 

(0.157) 

0. 

227 

(0. 

056) 

3.229 

(1.226) 

0 

275 

(0. 

Oil) 

0.09A 

(0.100) 

0 

226 

(0 

055) 

3.606 

(1.259) 

0 

301 

(0 

018) 

-1.  AA7 

(0.1A5) 

0 

224 

(0 

050) 

1.  OAA 

(0.223) 

0 

2A6 

(0 

047) 

0.048 

(0.097) 

0 

311 

(0 

038) 

-0.  181 

(0.086) 

0 

202 

(0 

050) 

2.368 

(0.598) 

0 

.271 

(0 

.  026) 

-0. 756 

(0.088) 

0 

.  191 

(0 

.  04A ) 

3.  858 

(0.878) 

0 

.242 

(0 

.  036) 

-0. 663 

(0.08A ) 

0 

.  185 

(0 

.  04A) 

0.  861 

(0 . 153) 

0 

.  210 

(0 

.  029) 

-0. 282 

(0.082) 

0 

.  196 

(0 

.047) 

-1.  28A 

(0. 118) 

0 

.  196 

(0 

.  046 ) 

2.  270 

( 0 . A  AO ) 

0 

.  10  4 

( 0 

.  012 ) 

2.  890 

(0.992) 

0 

.300 

(0 

.  013) 

2.  782 

(0.621) 

0 

.095 

(0 

.012) 

-0. 78A 

( 0. 113) 

0 

.228 

(0 

.  055; 

-0. 698 

(0.106) 

0 

.224 

(0 

.05A) 

2.  82A 

(0.805) 

0 

.229 

(0 

.  Oil) 

2.  653 

(0.637) 

0 

.  147 

(0 

.018) 

0.  065 

(0.078) 

0 

.200 

(0 

.031) 

0.  Oil 

(0.090) 

0 

.290 

(0 

.  036) 

0.  669 

(0. 130) 

0 

.221 

(0 

.026) 

1.  110 

(0.278) 

0 

.393 

(0 

.021) 

0.9A9 

(0.201) 

0 

.  185 

(0 

.015) 

-0. 336 

(0.076) 

0 

.  198 

(0 

.  040) 

-0 . 660 

(0.075) 

0 

.159 

(0 

.036) 
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Tablt  E.31 

1986  IRT  Parameters,  Mathematica  Trend  Items,  Age  9 


FIELD 

BLOCK 

ITEM 

N270901 

Ml 

1 

N277A01 

Ml 

2 

N267601 

Ml 

3 

N276801 

Ml 

N276802 

Ml 

5 

N276803 

Ml 

6 

N25070 1 

Ml 

7 

N250702 

Ml 

8 

N250703 

Ml 

9 

N262201 

Ml 

10 

N257201 

Ml 

11 

N276101 

Ml 

12 

N286101 

Ml 

13 

N270001 

Ml 

1^ 

N272102 

Ml 

15 

N284001 

Ml 

16 

N284002 

Ml 

17 

N267602 

Ml 

18 

N262501 

Ml 

19 

N262502 

Ml 

20 

N265401 

Ml 

21 

N266101 

Kl 

22 

N269101 

Ml 

23 

N268201 

Ml 

24 

N252101 

Ml 

25 

N272301 

M2 

1 

N276601 

M2 

2 

N257801 

M2 

3 

N263401 

M2 

N263402 

M2 

5 

N273501 

M2 

6 

N275^01 

M2 

7 

N277501 

M2 

8 

N277601 

M2 

9 

N277602 

M2 

10 

M2 

1 1 

JNZb  1^  0 1 

M2 

12 

U  O  C  A  C  A  1 

M2 

13 

\J  O  C  A  C  A  O 

M2 

14 

^rt  C  ft  *J 

r(z:>ubu  J 

M2 

1 5 

nz  0 1  *t  u  1 

M2 

16 

n  ZD  u  9  u 1 

nz 

1 7 

JCl  Z  D  U  9  U  Z 

MO 

nz 

1  o 

N2S0903 

M2 

19 

N276001 

M2 

21 

N276002 

M2 

22 

N271101 

M2 

24 

N252001 

M2 

25 

N26C001 

M2 

26 

N272801 

M3 

15 

N267001 

M3 

16 

N272101 

M3 

17 

N262401 

M3 

18 

N258501 

M3 

19 

A  SE  B 


0 

.  894 

(0 

.037) 

-2 

.  165 

1 

.  026 

(0 

.063) 

-1 

.573 

1 

.  268 

(0 

.066) 

-0 

.  611 

0 

.  490 

(0 

.045) 

-3 

.  763 

0 

.  725 

(0 

.038) 

-1 

.591 

0 

.  621 

(0 

.035) 

0 

.  147 

0 

.  743 

(0 

.044) 

-0 

.850 

1 

.  001 

(0 

.048) 

0 

.841 

1 

.054 

(0 

.064) 

0 

.OlS 

0 

.441 

(0 

.036) 

-1 

.218 

1 

.233 

(0 

084) 

"0 

.533 

0 

.963 

(0 

040) 

-0 

.758 

0 

814 

(0 

039) 

-0 

.521 

0 

448 

(0 

030) 

-0 

.727 

0 

.992 

(0 

062) 

0 

034 

0 

981 

(0 

050) 

-0 

383 

0 

792 

(0 

037) 

2 

.054 

1 

103 

(0 

057) 

-0 

074 

0 

269 

(0 

031) 

-0 

688 

0 

254 

(0 

062) 

6 

169 

1 

582 

(0 

164) 

2 

224 

0 

542 

(0 

052) 

1 

917 

0 

540 

(0 

071) 

2 

970 

1 

248 

(0 

058) 

1 

026 

0 

839 

(0 

060) 

1 

75Z 

0 

946 

(0 

052) 

-1 

947 

1 

061 

(0 

062) 

-1 

010 

0 

588 

(0 

038) 

-0 

909 

0 

888 

(0 

063) 

-0 

701 

1 

010 

(0 

080) 

-0 

203 

0 

744 

(0 

058) 

-0 

684 

0 

985 

(0 

043 ) 

-0 

478 

0 

842 

(0 

039) 

-0 

421 

1 

438 

(0 

049) 

-0 

522 

1 

267 

(0 

053) 

0 

172 

1 

507 

(0 

063 ) 

-0 

Oil 

0 

509 

(0 

042) 

-0 

145 

1 

097 

(0 

078) 

-0 

231 

0 

791 

(0 

053 ) 

-0 

584 

1 

366 

( 0 

071) 

0 

566 

A 

u 

ODH 

( 0 

042 ) 

~0 

265 

0 

599 

(0 

040) 

-0 

411 

1 

101 

(0 

051) 

1 

181 

0 

970 

(0. 

051) 

0 

685 

0 

879 

(0 

037) 

-0 

975 

0 

778 

(0. 

035) 

1 

507 

0 

626 

(0. 

034) 

-0 

305 

1 

244 

(0. 

131) 

2 

670 

0 

565 

(0. 

087) 

4. 

055 

0. 

576 

(0. 

049) 

-2. 

007 

0. 

597 

(0  . 

045) 

-1. 

392 

0. 

990 

(0. 

096) 

-0. 

533 

0. 

594 

(0  . 

069) 

0. 

928 

0. 

876 

(0. 

066) 

1. 

029 

SE  C  SE 


(0 

.098) 

0 

.0 

(0 

.0  ) 

(0 

.114) 

0 

.  177 

(0 

.038) 

(0 

.  049) 

0 

.  156 

(0 

.  020) 

(0 

.353  ) 

0 

0 

(0 

.0  ) 

(0 

.090  ) 

0 

0 

(0 

.0  ) 

(0 

.027  ) 

0 

0 

(0 

.0  ) 

(0 

.059) 

0 

.139 

(0 

.022) 

(0 

.054  ) 

0 

117 

(0 

oil) 

(0 

.033  ) 

0 

123 

(0 

016) 

(0 

.  105) 

0 

196 

(0 

024) 

(0 

.055) 

0 

283 

(0 

020) 

(0 

.042) 

0 

0 

(0 

0  ) 

(0 

.035) 

0 

0 

(0 

0  ) 

(0 

.053  ) 

0 

0 

(0 

0  ) 

(0 

.039) 

0 

173 

(0 

018) 

(0 

.033  ) 

0 

0 

(0 

0  ) 

(0 

103) 

0 

0 

(0 

0  ) 

(0 

031) 

0 

104 

(0 

014) 

(0 

084) 

0 

227 

(0 

019) 

(1 

519) 

0 

172 

(0 

008) 

(0 

3  60  ) 

0 

340 

(0 

Oil) 

(0 

192) 

0 

264 

(0 

Oil) 

(0 

402) 

0 

238 

(0 

009) 

(0 

068) 

0 

201 

(0 

010) 

(0 

143) 

0 

170 

(0 

012) 

(0 

123) 

0 

180 

(0 

040) 

(0 

076) 

0 

170 

(0 

029) 

(0 

066) 

0 

240 

(0 

022) 

(0 

063  ) 

0 

299 

(0 

022) 

(0 

043  ) 

0 

282 

(0 

018) 

(0 

0  68) 

0 

261 

(0 

026) 

(0 

033  ) 

0 

0 

(0 

0  ) 

(0 

031) 

0 

0 

(0 

0  ) 

(0 

037) 

0 

0 

(0 

0  ) 

(0 

029) 

0 

0 

(0 

0  ) 

(0 

030) 

0 

0 

(0 

0  ) 

(0 

037) 

0 

232 

(0 

020) 

(0 

045) 

0 

21?, 

(0 

019) 

(0 

054  ) 

0 

189 

(0 

023) 

(0 

056) 

0 

158 

(0 

013) 

( 0 

038) 

0 . 

151 

( 0 

021 ) 

(0 

040  ) 

0. 

178 

(0 

019) 

(0 

0  72) 

0. 

157 

(0 

010) 

(0 

050) 

0. 

109 

(0. 

012) 

(0 

04J  ) 

0. 

0 

(0. 

0  ) 

(0 

074) 

0. 

0 

(0. 

0  ) 

(0 

028) 

0. 

0 

(0. 

0  ) 

(0 

372) 

0. 

196 

(0. 

009) 

(0 

634  ) 

0. 

082 

(0. 

007) 

(0 

176) 

0. 

180 

(0. 

036) 

(0. 

110) 

0. 

249 

(0. 

026) 

(0 

071) 

0. 

286 

(0. 

024) 

(0 

116) 

0. 

300 

(0. 

013) 

(0 

092) 

0. 

236 

(0. 

012) 

-603- 
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Table  E.32 

1986  IRT  Parameters,  Mathematics  Trend  Items,  Age  13 


FIELD  BLOCK 

X 1  £.n 

A 

SE 

B 

SE 

SE 

N281901 

Ml 

1  c 

1 D 

0. 

925 

(0. 

OAO) 

-2. 

181 

(0. 

105) 

0 . 

lA  6 

(0. 

03A) 

N2SA601 

Ml 

1. 

092 

(0. 

05A) 

-1. 

553 

(0. 

089) 

0 . 

28A 

(0. 

030) 

N276801 

Ml 

17 

0. 

A33 

(0. 

0A9) 

-A  . 

715 

(0. 

5A2) 

n 

u . 

n 
u 

(0. 

0  ) 

N2768G2 

Ml 

1  fi 
1  O 

0. 

A93 

(0. 

OAA) 

-3. 

957 

(0. 

359) 

0 . 

0 

(0. 

0  ) 

N276803 

Ml 

1  Q 
19 

0. 

A35 

(0. 

033) 

-1. 

927 

(0. 

lAB) 

0 , 

(0. 

0  ) 

N277601 

Ml 

on 

0. 

856 

(0. 

036) 

-2. 

50A 

(0. 

113) 

0 . 

(0. 

0  ) 

N277602 

Ml 

Z 1 

0. 

62A 

(0. 

030) 

-1. 

885 

(0. 

095) 

0 . 

(0. 

0  ) 

N277603 

Ml 

o  o 

0. 

617 

(0. 

031) 

-2. 

267 

(0. 

117) 

0 . 

Q 

(0. 

0  ) 

N267201 

Ml 

Zo 

0. 

776 

(0. 

058) 

-1. 

051 

(0. 

087) 

0 , 

25A 

(0. 

026) 

N286201 

Ml 

2  A 

0. 

891 

(0. 

051) 

-0. 

892 

(0. 

061) 

n 

9  A  T 

(0. 

021) 

N250901 

Ml 

25 

0. 

A23 

(0. 

029) 

-2. 

565 

(0. 

176) 

n 

u . 

(0. 

027) 

N250902 

m 

ZD 

1. 

020 

(0. 

0A9) 

-0. 

3A9 

(0. 

031) 

0 . 

075 

(0. 

OlA) 

N250903 

Ml 

z  / 

0. 

820 

(0. 

039) 

-1. 

510 

(0. 

078) 

0  . 

096 

(0. 

025) 

N262401 

Ml 

9  ft 

0. 

85A 

(0. 

05A) 

-0. 

556 

(0. 

0A8) 

0 . 

323 

(0. 

017) 

N27A801 

Ml 

Z9 

0  . 

629 

(0. 

051) 

-0. 

192 

(0. 

036) 

0  ^ 

269 

(0. 

018) 

N265202 

Ml 

V  U 

0  . 

8A3 

(0. 

07A) 

-0. 

176 

(0. 

OAl) 

0  . 

339 

(0. 

018) 

N266801 

Ml 

O  1 

0. 

559 

(0. 

038) 

-1. 

108 

(0. 

080) 

0  . 

2A  8 

(0. 

021) 

N252901 

Ml 

o  z 

1. 

2A9 

(0. 

072) 

-0. 

036 

(0. 

033) 

0  . 

109 

(0. 

015) 

N262501 

HI 

3  3 

0. 

360 

(0. 

033) 

-0. 

237 

(0. 

03A) 

Q  ^ 

3  A  8 

(0  . 

015) 

N262502 

Ml 

O  *l 

1. 

216 

(0. 

068) 

1. 

97A 

(0. 

151) 

0  ^ 

3  7  9 

(0. 

008) 

N257601 

Ml 

O  D 

1. 

280 

(0. 

055) 

-0. 

538 

(0. 

035) 

0  ^ 

0 

(0. 

0  ) 

N265201 

Ml 

O  D 

0. 

810 

(0. 

062) 

-1. 

5A8 

(0. 

127) 

0  . 

339 

(0. 

032) 

N273901 

Ml 

O  / 

1. 

786 

(0. 

111) 

0. 

258 

(0. 

0A7) 

0  ^ 

18A 

(0. 

013) 

N2S8801 

Ml 

O  O 

1. 

273 

(0. 

055) 

1. 

12A 

(0. 

076) 

0 . 

397 

(0. 

010) 

N263101 

Ml 

T  Q 
O  9 

0. 

527 

(0. 

027) 

-0. 

291 

(0. 

02A) 

0  . 

0 

(0. 

0  ) 

N265901 

Ml 

L  n 

0. 

933 

(0. 

060) 

0. 

930 

(0. 

079) 

0 . 

333 

(0. 

012) 

N252101 

Ml 

J.  1 

0. 

933 

(0. 

056) 

0. 

623 

(0. 

05A) 

0 . 

2A  0 

(0. 

013) 

N275001 

Ml 

0. 

9A6 

(0. 

OAO) 

0. 

363 

(0. 

027) 

0  . 

0 

(0. 

0  ) 

N260101 

Ml 

L  T 
•1  o 

1. 

299 

(0. 

072) 

0. 

A15 

(0. 

0A2) 

0 . 

160 

(0. 

Oil) 

N269001 

Ml 

A  A 

1. 

012 

(0. 

053) 

0. 

3  82 

(0. 

036) 

0  . 

152 

(0. 

oil) 

N286301 

Ml 

M  D 

1. 

189 

(0. 

050) 

0. 

660 

(0. 

0A6) 

0 . 

205 

(0. 

010) 

N254602 

Ml 

A  6 

0. 

7AA 

(0. 

0A5) 

1. 

A13 

(0. 

095) 

0 . 

235 

(0. 

009) 

N261001 

Ml 

A  7 

0 

833 

(0. 

0A9) 

1 

Oil 

(0. 

070) 

0 

219 

(0 

010) 

N286501 

Ml 

A  8 

1 

256 

(0 

0A2) 

T 

161 

(0 

058) 

0 

lA  1 

(0 

008) 

N27890A 

Ml 

A  9 

1 

315 

(0 

057) 

1 

A87 

(0 

097) 

0 

19A 

(0 

010) 

N255701 

Ml 

50 

1 

317 

(0 

OAA  ) 

1 

268 

(0 

063) 

0 

139 

(0 

008) 

N283101 

Ml 

51 

1 

579 

(0 

0A9) 

1 

55A 

(0 

080) 

0 

lAB 

(0 

006) 

N277A01 

M2 

Q 

o 

0 

778 

(0 

056) 

-2 

.903 

(0 

220) 

0 

.  lA 5 

(0 

0A2) 

N277901 

M2 

g 

0 

.591 

(0 

033) 

-3 

.506 

(0 

199) 

0 

.  0 

(0 

.0  ) 

N277902 

M2 

10 

0 

.688 

(0 

.036) 

-3 

.301 

(0 

178) 

0 

.  0 

(0 

.0  ) 

N277903 

M2 

j_  ^ 

0 

.573 

(0 

.030) 

-2 

.859 

(0 

.  15A) 

0 

.  0 

(0 

.0  ) 

N263401 

M2 

12 

0 

.675 

(0 

.0A6) 

-2 

.751 

(0 

.  196) 

0 

.  257 

(0 

.OAO) 

N263402 

M2 

1  o 

0 

.635 

(0 

.0A5) 

-2 

.  A78 

(0 

.  181) 

0 

.  263 

(0 

.036) 

K':5n701 

M2 

1  A 

0 

.688 

(0 

.035) 

-2 

.717 

(0 

.  1A3) 

0 

.  106 

(0 

.033) 

N250702 

M2 

1  D 

1 

.1A5 

(0 

.051) 

-0 

.797 

(0 

.0A7) 

Q 

.  102 

(0 

.018) 

N250703 

M2 

1  D 

0 

.6A9 

(0 

.031) 

-2 

.  110 

(0 

.  106) 

0 

.  110 

(0 

.028) 

N256101 

M2 

1  / 

0 

.760 

(0 

.033) 

-1 

.056 

(0 

.052) 

Q 

.  0 

(0 

.0  ) 

N262201 

M2 

1  Q 

0 

.520 

(0 

.037  ) 

-1 

.789 

(0 

.  132) 

Q 

.361 

(0 

.023) 

N270301 

M2 

0 

.A21 

(0 

.031) 

-1 

.596 

(0 

.  119) 

0 

.  126 

(0 

.022) 

N270302 

M2 

2 1 

1 

.018 

(0 

.0A7) 

2 

.  19A 

(0 

.  118) 

0 

.  051 

(0 

.005) 

N253701 

M2 

22 

0 

.361 

(0 

.031) 

-0 

.50'* 

(0 

.050) 

0 

.  27 1 

(0 

.016) 

N286601 

M2 

1 

.698 

(0 

.059) 

-0 

.  19A 

(0 

.  029) 

0 

.  0 

(0 

.0  ) 

N286602 

M2 

2  A 

1 

.363 

(0 

.051) 

-0 

.2A7 

(0 

.  027) 

0 

.  0 

(0 

.0  ) 

N286603 

M2 

25 

1 

.  A9A 

(0 

.050) 

0 

.  A05 

(0 

.030) 

0 

.  0 

(0 

.0  ) 

N269101 

M2 

26 

0 

.7  52 

(0 

.A8) 

-0 

.38A 

(0 

.037) 

0 

.213 

(0 

.016) 

N282201 

M2 

28 

1 

.063 

(0 

.058) 

0 

.576 

(0 

.051) 

0 

.  3  A3 

(0 

.011) 

N278902 

M2 

29 

0 

.720 

(0 

.051) 

1 

.338 

(0 

.  107) 

0 

.216 

(0 

.012) 

N263501 

M2 

30 

1 

.389 

(0 

.092) 

0 

.  187 

(0 

.036) 

0 

.  115 

(0 

.012) 

N258802 

M2 

3 1 

1 

.619 

(0 

.078) 

0 

.  ABA 

(0 

.051) 

0 

.  25A 

(0 

.011) 

N278901 

M2 

32 

1 

.559 

(0 

.086) 

0 

.  A15 

(0 

.051) 

0 

.212 

(0 

.013) 

N26A701 

M2 

33 

1 

.  175 

(0 

.056) 

0 

.867 

(0 

.059) 

0 

.206 

(0 

.010) 

N261501 

M2 

3A 

0 

.661 

(0 

.056) 

-0 

.5A5 

(0 

.055) 

0 

.  lAl 

(0 

.020) 

N261801 

M2 

35 

0 

.679 

(0 

.053) 

0 

.  OAA 

(0 

.033) 

0 

.223 

(0 

.017) 

N261601 

M2 

36 

0 

.3AA 

(0 

.0A3) 

1 

.903 

(0 

.239) 

0 

.  155 

(0 

.012) 

N261301 

M2 

37 

0 

.700 

(0 

.0A8) 

0 

.768 

(0 

.062) 

0 

.  113 

(0 

.012) 

N261201 

M2 

38 

0 

.525 

(0 

.052) 

1 

.619 

(0 

.166) 

0 

.219 

(0 

.012) 

N281A01 

M2 

39 

0 

.728 

(0 

.050) 

1 

.711 

(0 

.  127) 

0 

.  106 

(0 

.009) 

N252001 

M2 

AO 

1 

.  A23 

(0 

.06A  ) 

0 

.832 

(0 

.062) 

0 

.  179 

(0 

.010) 

N258803 

M2 

Al 

1 

.  191 

(0 

.OAA) 

1 

.351 

(0 

.068) 

0 

.  170 

(0 

.007) 

N278903 

M2 

A2 

1 

.338 

(0 

.058) 

1 

.066 

(0 

.073) 

0 

.  169 

(0 

.010) 

N286502 

M2 

A3 

1 

.671 

(0 

.05A) 

1 

.  171 

(0 

.068) 

0 

.160 

(C 

.008) 

N275301 

M3 

25 

0 

.372 

(0 

.028) 

.723 

(0 

.  132) 

Q 

.  1A7 

(0 

.0  12) 

N282202 

M3 

26 

0 

.936 

(0 

.066) 

-c 

.  A58 

(0 

.0A5) 

c 

.255 

(C 

.o;7) 

Table  E.32 
( continuad) 


FIELD 

BLOCK 

ITEM 

A 

SE 

B 

SE 

C 

SE 

N266101 

M3 

27 

0 

849 

CO 

.065) 

-0 

161 

CO 

.033) 

0 

292 

(0 

.014) 

N254001 

M3 

28 

1 

161 

(0 

.084) 

-0 

A79 

CO 

.047) 

0 

118 

CO 

.017) 

N26ggoi 

M3 

29 

0 

66A 

(0 

.0A9) 

-0 

27A 

CO 

.035) 

0 

288 

(0 

.015) 

N256501 

M3 

30 

0 

866 

(0 

.069) 

0 

581 

CO 

.061) 

0 

318 

CO 

.012) 

N265902 

M3 

31 

1 

077 

CO 

.073) 

1 

170 

CO 

.  103) 

0 

328 

(0 

.011) 

N256801 

M3 

32 

1 

051 

CO 

.069} 

0 

841 

CO 

.072) 

0 

312 

CO 

.  Oil) 

ERIC 


-605- 

651 


Table  E.33 

1986  IRT  Parameters,  Mathematics  Trend  Items,  Age  17 


FIELD  BLOCK 

ITEM 

S£ 

B 

SE 

c 

SE 

N2S6101 

Ml 

15 

1  ^ 

oil 

(0. 

035) 

-1. 

769 

(0. 

071) 

0  . 

0 

(0. 

0  ) 

N260601 

Ml 

16 

1 , 

295 

(0. 

035) 

"1. 

305 

(0. 

049) 

0  . 

0 

(0. 

0  ) 

N262A01 

Ml 

17 

0  . 

832 

(0. 

040) 

-1. 

129 

(0. 

063) 

0  . 

24  4 

(0. 

023) 

N258804 

Ml 

18 

0 . 

524 

(0. 

038) 

-1. 

545 

(0. 

115) 

0  . 

282 

(0. 

024) 

N286001 

Ml 

19 

0  . 

827 

(0. 

043) 

-0. 

908 

(0. 

055) 

0  . 

187 

(0. 

021) 

N286002 

Ml 

20 

1 , 

370 

(0. 

048) 

-1. 

234 

(0. 

060) 

0  . 

151 

(0. 

027) 

N286302 

Ml 

22 

1  ^ 

4  7  4 

(0. 

071) 

-0. 

297 

(0. 

042) 

0  . 

300 

(0. 

016) 

N278501 

Ml 

23 

1 . 

405 

(0. 

042) 

-0. 

803 

(0. 

038) 

0  . 

0 

(0. 

0  ) 

N278502 

Ml 

2^ 

1 . 

364 

(0. 

050) 

-0. 

464 

(0. 

032) 

0  . 

0 

(0. 

0  ) 

N278503 

Ml 

25 

1  ^ 

252 

(0. 

039) 

-0. 

755 

(0. 

036) 

0  . 

0 

(0. 

0  ) 

N258802 

Ml 

26 

1  ^ 

429 

(0. 

087) 

-0  . 

343 

(0. 

043) 

0  . 

278 

(0. 

016) 

N25A602 

Ml 

27 

1  ^ 

607 

(0. 

087) 

-0. 

365 

(0. 

042) 

0  . 

214 

(0. 

016) 

N259901 

Ml 

28 

0  . 

943 

(0. 

C50) 

-0. 

568 

(0. 

043) 

0  . 

218 

(0. 

018) 

N287101 

Ml 

29 

1  ^ 

956 

(0. 

101) 

-0  . 

319 

(0. 

043) 

0  . 

218 

(0. 

014  ) 

N270301 

Ml 

30 

0  . 

987 

(0. 

041) 

-1. 

293 

(0. 

064  ) 

0  . 

17  8 

(0. 

02'j) 

N270302 

Ml 

31 

1  ^ 

865 

(0. 

093) 

0. 

262 

(0. 

040) 

0  . 

093 

(0. 

010) 

N255701 

Ml 

32 

1  ^ 

393 

(0. 

090) 

-0. 

381 

(0. 

044  ) 

0  . 

201 

(0. 

017) 

•N25^»301 

Ml 

33 

1  ^ 

100 

(0. 

065) 

0. 

007 

(0. 

033) 

0  . 

267 

(0. 

013) 

N286502 

Ml 

3A 

2  . 

315 

(0. 

106) 

-0. 

185 

(0. 

039) 

0  . 

079 

(0. 

Oil) 

N260901 

Ml 

35 

622 

(0. 

096) 

-0. 

066 

(0. 

034) 

0  . 

148 

(C. 

013) 

H256801 

Ml 

36 

1 . 

059 

(0. 

062) 

-0. 

389 

(0. 

040) 

0  . 

242 

(0. 

017) 

N258803 

Ml 

37 

1 . 

231 

(0. 

057) 

0. 

468 

(0. 

042) 

0  . 

221 

(0. 

010) 

N262601 

Ml 

38 

0  . 

723 

(0. 

048) 

0. 

4  52 

(0. 

043) 

0  . 

220 

<o. 

012) 

N253901 

Ml 

39 

1 . 

643 

(0. 

110) 

-0. 

110 

(0. 

041) 

0  . 

256 

(0. 

014  ) 

N2S3902 

Ml 

AO 

0  . 

650 

(0. 

047  ) 

0. 

527 

(0. 

053) 

0  . 

374 

(0. 

013) 

N253903 

Ml 

A  1 

1 . 

226 

(0. 

061) 

0. 

608 

(0. 

053) 

0  . 

297 

(0. 

Oil) 

N2S390A 

Ml 

A2 

1 

725 

(0. 

088) 

0. 

461 

(0. 

056) 

0  . 

324 

(0. 

Oil) 

N263001 

Ml 

A  3 

0 

574 

(0. 

028) 

0. 

367 

(0. 

027  ) 

0  . 

0 

(0. 

0  ) 

N278905 

Ml 

1 

116 

(0. 

056) 

1 

169 

(0. 

084  ) 

0 

247 

(0 

oil) 

N287301 

Ml 

4  5 

0 

738 

(0. 

030) 

-0 

363 

(0 

025) 

0 

0 

(0 

0  ) 

N287302 

Ml 

^^  6 

0 

700 

(0 

029) 

1 

442 

(0 

065) 

0 

0 

(0 

0  ) 

N26A301 

Ml 

A  7 

Q 

671 

(0 

029) 

1 

008 

(0 

049) 

0 

0 

(0 

0  ) 

N282801 

Ml 

A  8 

1 

7  62 

(0 

057) 

1 

427 

(0 

086) 

0 

168 

(0 

008) 

N251101 

Ml 

4  9 

1 

132 

(0 

036) 

1 

268 

(0 

051) 

0 

0 

(0 

0  ) 

N25A601 

M2 

15 

1 

170 

(0 

045) 

-2 

100 

(0 

100) 

0 

242 

(0 

042) 

N262301 

M2 

17 

0 

4  65 

(0 

037) 

-1 

753 

(0 

141) 

0 

268 

(0 

025) 

N263201 

M2 

18 

1 

037 

(0 

052) 

-1 

051 

(0 

066) 

0 

358 

(0 

022) 

N263202 

M2 

19 

1 

.  199 

(0 

090) 

-0 

231 

(0 

044) 

0 

435 

(0 

014) 

N260101 

M2 

20 

1 

.418 

(0 

054) 

-0 

857 

(0 

049) 

0 

180 

(0 

021) 

N25A001 

M2 

21 

0 

.  961 

(0 

044  ) 

-0 

786 

(0 

046) 

0 

188 

(0 

019) 

N269001 

M2 

22 

0 

.  903 

(0 

079) 

-0 

.271 

(0 

.043) 

0 

.420 

(0 

.009) 

N278901 

M2 

23 

0 

,961 

(0 

.065) 

-0 

.609 

(0 

.057) 

0 

.  293 

(0 

.022) 

N261501 

M2 

24 

0 

.  94  1 

(0 

.050) 

-1 

.360 

(0 

.  083) 

0 

.  189 

(0 

.  030  ) 

N261801 

M2 

25 

0 

.  590 

(0 

.  045) 

-1 

.011 

(0 

.  085) 

0 

.234 

(0 

.  024  ) 

N261201 

M2 

26 

0 

.  579 

(0 

.  046) 

0 

.  159 

(0 

.  035) 

0 

.205 

(0 

.016) 

N261601 

M2 

27 

0 

.456 

(0 

.043) 

1 

.  128 

(0 

.  114  ) 

0 

.219 

(0 

.  013 ) 

N261301 

M2 

28 

0 

.662 

(0 

.047) 

0 

.  530 

(0 

.051) 

0 

.  167 

(0 

.014  ) 

N281A01 

M2 

29 

0 

.  680 

(0 

.  046) 

1 

.303 

(0 

.099) 

0 

.  176 

(0 

.  Oil ) 

N280401 

M2 

30 

0 

.  638 

(0 

.028) 

-1 

.099 

(0 

.053) 

0 

.  0 

(0 

.0  ) 

N259001 

M2 

31 

0 

.  804 

(0 

.031) 

-0 

.725 

(0 

.036) 

0 

.0 

(0 

.0  ) 

N287102 

M2 

32 

1 

.  992 

(0 

.088) 

-0 

.493 

(0 

.046) 

0 

.  209 

(0 

.016) 

N286301 

M2 

33 

1 

.468 

(0 

.045) 

-0 

.431 

(0 

.  033) 

0 

.  123 

(0 

.014  ) 

N286501 

M2 

34 

2 

.287 

(0 

.081) 

-0 

.380 

(0 

.043) 

0 

.  136 

(0 

.013 ) 

N262501 

M2 

35 

0 

.449 

(0 

.037) 

-0 

.340 

(0 

.040) 

0 

.373 

(0 

.015) 

N262502 

M2 

36 

1 

.071 

(0 

.063) 

1 

.342 

(0 

.  105) 

0 

.462 

(0 

.009) 

N263101 

M2 

37 

0 

.  671 

(0 

.030) 

-0 

.710 

(0 

.039) 

0 

.  0 

(0 

.0  ) 

N258801 

M2 

38 

0 

.  991 

(0 

.058) 

-0 

.207 

(0 

.034) 

0 

.264 

(0 

.015) 

N26^»701 

M2 

39 

1 

.396 

(0 

.069) 

-0 

.259 

(0 

.037  ) 

0 

.216 

(0 

.015) 

N261001 

M2 

4  0 

0 

.741 

(0 

.045) 

-0 

.322 

(0 

.033) 

0 

.213 

(0 

.016) 

N251701 

M2 

I^  1 

1 

.  168 

(0 

.067) 

0 

.042 

(0 

.039) 

0 

.  186 

(0 

.  016 ) 

N278902 

M2 

42 

1 

.  286 

(0 

.096) 

0 

.  148 

(0 

.048) 

0 

.286 

(0 

.015) 

N260801 

M2 

43 

1 

.453 

(0 

.061) 

0 

.  148 

(0 

.027) 

0 

.0 

(0 

.0  ) 

N278903 

M2 

44 

1 

.287 

(0 

.082) 

0 

.  138 

(0 

.043) 

0 

.  196 

(0 

.015) 

N255601 

M2 

45 

0 

.896 

(0 

.057) 

1 

.  867 

(0 

.141) 

0 

.366 

(0 

.010) 

N255301 

M2 

46 

1 

.375 

(0 

.054) 

1 

.594 

(0 

.096) 

0 

.259 

(0 

.008) 

N268901 

H2 

47 

2 

.090 

(0 

.073) 

0 

.679 

(0 

.058) 

0 

.  175 

(0 

.009) 

N268801 

M2 

48 

1 

.447 

(0 

.046) 

1 

.292 

(0 

.065) 

0 

.097 

(0 

.007  ) 

N255801 

M2 

49 

0 

.870 

(0 

.030) 

1 

.800 

(0 

.070) 

0 

.0 

(0 

.0  ) 

N266501 

M3 

31 

0 

.678 

(0 

.051) 

-0 

.532 

(0 

.049) 

0 

.248 

(0 

.018) 

N271301 

M3 

32 

1 

.416 

(0 

.  116) 

0 

.349 

(0 

.056) 

0 

.290 

(0 

.012) 

N255501 

M3 

33 

0 

.842 

(0 

.067) 

0 

.436 

(0 

.052) 

0 

.316 

(0 

.013> 

N256001 

M3 

34 

0 

.836 

(0 

.050) 

0 

.  106 

(0 

.024) 

0 

.0 

(0 

.0  ) 

N257101 

M3 

35 

0 

.438 

(0 

.058) 

2 

.771 

(0 

.372) 

0 

.298 

(0 

.010) 

-606- 


ERIC 


65. 


1986 


FIELD 

ITEM 

N400001 

SI 

6 

NA00301 

SI 

8 

N400A01 

SI 

g 

NA00A02 

SI 

10 

N400403 

SI 

11 

N40040A 

SI 

12 

NA00405 

SI 

13 

N400501 

SI 

lA 

N400101 

SI 

15 

N400102 

SI 

16 

N400601 

SI 

17 

N400701 

SI 

18 

NAOOgOl 

SI 

ig 

N401001 

SI 

20 

NAOllOl 

SI 

21 

NA01201 

SI 

22 

N401301 

SI 

23 

NA01501 

S2 

1 

NA01601 

S2 

2 

N401702 

S2 

A 

N401703 

S2 

5 

N401801 

S2 

6 

N401802 

S2 

7 

N401803 

S2 

S 

N401804 

S2 

g 

NAOigOl 

S2 

10 

N402001 

S2 

11 

N402002 

S2 

12 

N402005 

S2 

15 

N4021U1 

S2 

16 

N402201 

S2 

17 

N402401 

S2 

18 

N402501 

S2 

ig 

N402602 

S2 

21 

N402701 

S2 

23 

N402801 

S2 

2A 

NA02g01 

S2 

25 

NA03001 

S3 

12 

NA03101 

S3 

13 

NA03201 

S3 

lA 

NA03202 

S3 

15 

NA03301 

S3 

16 

NA03A01 

S3 

17 

NA03501 

S3 

18 

NA03502 

S3 

ig 

NA03503 

S3 

20 

NA03601 

S3 

21 

NA03701 

S3 

22 

NA03702 

S3 

23 

NA03703 

S3 

2A 

NA03801 

S3 

25 

NA03803 

S3 

27 

NA0380A 

S3 

28 

NA03g01 

S3 

2g 

NAOAOOl 

S3 

30 

NA0A201 

S3 

31 

ERIC 


TabI*  £.3A 

IRT  Paratnaters,  Scitnce  Trend  Itams,  Agt  9 


A 

SE 

fi 

SE 

c 

SE 

0 

egg 

(0 

og3) 

-0 

7g3 

(0 

118) 

0 

218 

CO 

.053) 

0 

725 

(0 

ogg ) 

-0 

og3 

(0 

121) 

0 

323 

CO 

.052) 

0 

8gg 

(0 

121) 

-1 

AOO 

(0 

22g) 

0 

A68 

CO 

.067) 

1 

5g2 

(0 

177 ) 

-0 

5gA 

(0 

122) 

0 

353 

CO 

.0A3) 

0 

508 

(0 

08A ) 

-2 

237 

(0 

388) 

0 

A70 

CO 

.071) 

1 

162 

(0 

13A ) 

-0 

A30 

(0 

112) 

0 

.355 

(0 

.0A5) 

0 

818 

( 0 

log) 

-0 

8A3 

(0 

163) 

0 

AAO 

(0 

.061) 

0 

530 

( 0 

og5 ) 

0 

558 

(0 

188) 

0 

33A 

CO 

.05A) 

0 

58g 

( 0 

193 ) 

2 

037 

(0 

767) 

0 

531 

CO 

.033) 

0 

8Ag 

( 0 

202 ) 

1 

3g2 

(0 

A56) 

0 

A55 

CO 

.030) 

0 

620 

(0 

075) 

-0 

ogA 

(0 

og8) 

0 

1S7 

CO 

.050) 

0 

630 

( 0 

066 ) 

0 

068 

(0 

088) 

0 

18A 

CO 

.0A2) 

0 

228 

( 0 

0A8) 

2 

692 

(0 

581) 

0 

210 

CO 

.OAl) 

0 

A7A 

( 0 

063 ) 

0 

6A8 

(0 

137) 

0 

181 

CO 

.OAA) 

0 

270 

(0 

063) 

1 

807 

(0 

AAg) 

0 

227 

CO 

.056) 

0 

750 

(0 

238) 

2 

672 

(0 

0 

278 

CO 

.022) 

0 

527 

( 0 

117 ) 

1 

63A 

(0 

atT) 

0 

220 

CO 

.OAl) 

0 

288 

( 0 

056) 

-0 

52g 

(0 

162) 

0 

3Ag 

CO 

068) 

0 

5g6 

(0 

06A ) 

-1 

008 

(0 

131) 

0 

172 

CO 

.051) 

0 

37A 

( 0 

096 ) 

0 

8*5 

(0 

308) 

0 

508 

CO 

.059) 

0 

323 

( 0 

086 ) 

1 

157 

(0 

376) 

0 

A88 

CO 

.05g) 

0 

7A?. 

( 0 

120 ) 

-0 

028 

(0 

150) 

0 

A37 

CO 

.053) 

0 

6A6 

( 0 

101 ) 

~o 

650 

(0 

172) 

0 

A72 

CO 

.  06A) 

0 

Ag3 

( 0 

ogA ) 

"0 

020 

(0 

165) 

0 

A61 

CO 

.062) 

0 

A72 

( 0 

11 A ) 

1 

og7 

( 0 

3AA ) 

0 

A3g 

CO 

.  052) 

0 

506 

( 0 

125 ) 

1 

381 

( 0 

A12) 

0 

363 

CO 

.OAg) 

1 

03A 

( 0 

132 ) 

-0 

7A7 

(0 

1A8) 

0 

A2g 

CO 

.056) 

1 

015 

( 0 

130 ) 

-0 

ggA 

( 0 

173 ) 

0 

A38 

CO 

.060  ) 

0 

go6 

( 0 

125 ) 

-0 

3g7 

(0 

136) 

0 

A30 

{0 

.053 ) 

0 

A82 

( 0 

063 ) 

0 

307 

( 0 

112) 

0 

ig2 

CO 

.050) 

0 

357 

(0 

058) 

0 

317 

(0 

120) 

0 

20A 

CO 

.055) 

0 

ig8 

(  0 

0  AA ) 

3 

003 

(0 

680 ) 

0 

218 

CO 

.  OAl) 

0 

372 

( 0 

116 ) 

3 

363 

( 1 

085) 

0 

221 

CO 

.036) 

0 

3g3 

(  0 

073 ) 

-0 

g7i 

( 0 

228) 

0 

A7A 

CO 

.  06g) 

0 

522 

(0 

120 ) 

1 

g5A 

(0 

Ag8) 

0 

ig6 

CO 

.037) 

0 

582 

( 0 

18g ) 

3 

1A5 

( 1 

ogi) 

0 

180 

CO 

.  023) 

0 

A05 

( c 

lA  A ) 

A 

516 

( 1 

6A5) 

0 

168 

CO 

.  023 ) 

0 

671 

(0 

og6) 

-2 

88A 

(0 

A31) 

0 

201 

CO 

.057) 

0 

63g 

(0 

088) 

-2 

688 

(0 

386) 

0 

20A 

CO 

.058) 

0 

688 

( 0 

077 ) 

-2 

008 

(0 

2A3) 

0 

182 

CO 

05A) 

0 

A50 

( 0 

055 ) 

-1 

12A 

(0 

159) 

0 

ig5 

CO 

055) 

0 

700 

( 0 

075 ) 

-0 

767 

(0 

116) 

0 

200 

CO 

.053) 

c 

A12 

U OZ  J 

0 

688 

( 0 

210) 

0 

332 

C  0 

.  058) 

0 

722 

(0 

130) 

0 

AOO 

(0 

188) 

0 

A26 

CO 

.048) 

0 

678 

(0 

ogA) 

-1 

627 

CO 

255) 

0 

AAl 

CO 

067) 

0 

A76 

(0 

08g) 

-0 

025 

CO 

158) 

0 

4A7 

CO 

.062) 

0 

676 

(0 

103) 

0 

562 

CO 

161) 

0 

262 

CO 

0A3) 

2 

672 

(0 

286) 

-0 

178 

CO 

og7) 

0 

3go 

CO 

025) 

2 

116 

(0 

205) 

-0 

376 

CO 

ogA) 

0 

33g 

CO 

030) 

2 

121 

(0 

20A) 

-0 

ig6 

CO 

087) 

0 

337 

CO 

.027) 

0 

AA2 

(0 

128) 

1 

5gg 

CO 

53A) 

0 

A86 

CO 

0A8) 

0 

5A0 

(0 

085) 

-0 

660 

CO 

167) 

0 

AAO 

CO 

065) 

0 

657 

(0 

100) 

-0 

352 

(0 

lAA) 

0 

A28 

CO 

05g) 

0 

760 

(0 

072) 

-0 

2A5 

CO 

083) 

0 

ig5 

CO 

0A3) 

0 

257 

(0 

0A3) 

1 

08A 

CO 

206) 

0 

17g 

CO 

0A8) 

0 

37g 

(0 

076) 

1 

A6A 

CO 

331) 

0 

187 

CO 

OAg) 

-607- 


Table  E.35 

1986  IRT  Parameters.   Science  Trend  Items,  Age  13 


FIELD  BLOCK 

ITEM 

A 

SE 

B 

SE 

c 

SE 

NAOASOl 

SI 

12 

1 . 

348 

(0. 

039) 

-2 . 

091 

(0. 

082) 

0  . 

182 

(0. 

037) 

NA0A6O1 

SI 

13 

0 . 

420 

(0. 

023) 

-1 . 

199 

(0. 

069) 

0  . 

270 

(0. 

013) 

NA0A701 

SI 

14 

0 . 

684 

(0. 

034  ) 

- 1 . 

687 

(0. 

090) 

0 . 

216 

(0. 

025) 

NA0A702 

SI 

15 

0 . 

592 

(0. 

046) 

-0 . 

304 

(0. 

037  ) 

0 . 

23 1 

(0. 

015) 

NA00201 

SI 

16 

0 . 

425 

(0. 

024) 

- 1  ^ 

954 

(0. 

114) 

0 . 

229 

(0. 

016) 

NA0A901 

SI 

17 

0 . 

868 

(0. 

039) 

_Q  ^ 

(0. 

050) 

0  . 

23 1 

(0. 

017) 

NA0A801 

SI 

20 

1 

X  . 

U  JO 

(0. 

053) 

-  9 

X  9  0 

(0. 

130) 

n 

u . 

A  n  ^ 

•4  U  J 

(0. 

041) 

NA0A802 

SI 

21 

1 

X  . 

7  90 

(0. 

076) 

-n 

00  9 

(0. 

065) 

0 . 

34  9 

(0. 

019) 

NA0A803 

SI 

22 

1 

X  . 

19  7 

(0. 

100) 

—  n 

9  7 

(0. 

046) 

n 

u  . 

^ 

0  0  D 

(0. 

014) 

NA05001 

SI 

23 

n 
u . 

fifi 
O  OO 

(0. 

021) 

_  Q 

04  5 

(0. 

015) 

n 

u  . 

9n  «\ 

(0. 

009) 

NA05101 

SI 

9  L 
*4 

n 

/  OO 

(0. 

029) 

u  . 

1  ^  n 

(0. 

022) 

n 

u  . 

9  1  K 
Z  X  D 

(0. 

007  ) 

NA05201 

SI 

25 

0 . 

374 

(0. 

021) 

-  0  . 

205 

(0. 

019) 

0  . 

22,3 

(0. 

010) 

NA05301 

SI 

26 

0 . 

929 

10. 

105) 

0 . 

582 

(0. 

078) 

0 . 

243 

(0. 

010) 

NA05A01 

SI 

27 

0 . 

7  69 

(0. 

029) 

0  . 

821 

(0. 

038) 

0  . 

166 

(0. 

007) 

NA01201 

SI 

28 

0 . 

707 

(0. 

030) 

0  . 

014 

(0. 

019) 

0  . 

24  2 

(0. 

009) 

NA05501 

SI 

29 

0 . 

497 

(0. 

027) 

0 . 

813 

(0. 

048) 

0  . 

208 

(0. 

008) 

NA05601 

SI 

30 

0 . 

215 

(0. 

038) 

1 . 

799 

(0. 

320) 

0  . 

224 

(0. 

012) 

NA05701 

SI 

31 

1  ^ 

243 

(0. 

129) 

0  . 

290 

(0. 

050) 

0  . 

183 

(0. 

Oil) 

NA05801 

SI 

32 

0 . 

711 

(0. 

083) 

1  ^ 

022 

(0. 

126) 

0  . 

163 

(0. 

009) 

NA05901 

SI 

33 

0 . 

87  8 

(0. 

037) 

1  ^ 

139 

(0. 

059) 

0  . 

221 

(0. 

007  } 

NA06001 

SI 

34 

0 . 

926 

(0. 

039) 

2 . 

078 

(0. 

100) 

0 . 

226 

(0. 

005) 

NA06101 

SI 

35 

0 . 

689 

(0. 

029) 

2 . 

085 

(0. 

094  ) 

0 . 

185 

(0. 

005) 

NA06201 

SI 

36 

1  ^ 

129 

(0. 

033) 

2 . 

082 

(0. 

077  ) 

0  . 

133 

(0. 

004) 

NA06301 

S2 

10 

0 . 

237 

(0. 

023) 

-3  ^ 

920 

(0. 

386) 

0  . 

432 

(0. 

016) 

NA06302 

S2 

\  \ 

0 . 

435 

(0. 

027) 

-Q  ^ 

015 

(0. 

021) 

0 . 

43  4 

(0. 

009) 

NA06303 

S2 

12 

0  ^ 

600 

(0. 

030) 

n 

0  DU 

(0. 

028) 

n 

TOT 

(0. 

008) 

NA0630A 

S2 

13 

Q 

4  60 

(0. 

027) 

—  n 

n  A  T 
u  1 0 

(0. 

020) 

n 

u  . 

Q  7  7 
0  /  / 

(0. 

009) 

NA06A01 

S2 

14 

0 . 

486 

(0. 

031) 

u . 

9  Q  R 
ZOO 

(0. 

026) 

n 

A  7 1 
•4  /  X 

(0. 

009) 

NA06A02 

S2 

1 5 

0 . 

624 

(0 

030) 

-  n 

0  D  9 

(0 

027) 

n 
u 

Q 

0  0  0 

(0. 

Oil) 

NA06403 

S2 

16 

0 . 

722 

(0 

032) 

—  1 

X 

•i  «\9 

(0. 

074  ) 

Q 

411 

(0. 

017) 

NA06A0A 

S2 

\^ 

0 

966 

(0 

047) 

-  0 

4  81 

(0 

034) 

Q 

A26 

(0 

Oil) 

NA06A05 

S2 

18 

0 

613 

(0 

031) 

- 1 

0  14 

(0 

056) 

0 

3  87 

(0 

014) 

NA06501 

S2 

19 

0 

670 

(0 

045) 

"0 

317 

(0 

032) 

0 

203 

(0 

013) 

NA06601 

S2 

20 

0 

376 

(0 

021) 

-1 

155 

(0 

066) 

0 

202 

(0 

012) 

NA06701 

S2 

21 

0 

583 

(0 

036) 

-0 

492 

(0 

038) 

0 

223 

(0 

013) 

N406801 

S2 

22 

0 

928 

(0 

033) 

-1 

612 

(0 

065) 

0 

357 

(0 

020) 

N4068^ 

S2 

23 

0 

474 

(0 

057) 

3 

024 

(0 

367) 

0 

548 

(0 

005) 

N406803 

S2 

24 

0 

835 

(0 

031) 

-1 

080 

(0 

046) 

0 

315 

(0 

014) 

N40680A 

S2 

25 

0 

726 

(0 

028) 

-1 

292 

(0 

055) 

0 

.300 

(0 

015) 

N406805 

S2 

26 

1 

259 

(0 

059) 

1 

575 

(0 

106) 

0 

.611 

(0 

005) 

N406806 

S2 

27 

0 

326 

(0 

023) 

-0 

.  114 

(0 

.020) 

0 

.349 

(0 

.010) 

N406901 

S2 

28 

0 

526 

(0 

.024) 

-0 

.  296 

(0 

.021) 

0 

.  201 

(0 

.010) 

NA07001 

S2 

29 

0 

.  280 

(0 

.021) 

-0 

.  069 

(0 

.017) 

0 

.  206 

(0 

.010) 

NA07101 

S2 

30 

0 

.  998 

(0 

.032) 

1 

.  561 

(0 

.062) 

0 

.  140 

(0 

.006) 

NA07201 

S2 

31 

0 

.414 

(0 

.025) 

0 

.  556 

(0 

.037) 

0 

.  229 

(0 

.008) 

NA07301 

S2 

32 

0 

.  316 

(0 

.027) 

1 

.903 

(0 

.  165) 

0 

.  259 

(0 

.008) 

NA07302 

S2 

33 

0 

.  27 1 

(0 

.026) 

1 

44  8 

(0 

.  143) 

Q 

.277 

(0 

.009* 

NA08001 

S2 

34 

0 

.  991 

(0 

.112) 

Q 

.474 

(0 

.064) 

Q 

.23  7 

(0 

.OO81 

NA07601 

S2 

35 

0 

.  712 

(0 

.  101) 

0 

.697 

(0 

.  106) 

Q 

.198 

(0 

.011) 

NA07701 

S2 

37 

0 

.  456 

(0 

.023) 

0 

.  642 

(0 

.037  ) 

0 

.  164 

(0 

.008) 

NA07801 

S2 

38 

0 

.  707 

(0 

.038) 

1 

.  721 

(0 

.098) 

0 

241 

(0 

.006) 

NA07901 

S2 

39 

0 

.  428 

(0 

.039) 

0 

.386 

(0 

.043) 

0 

.  190 

(0 

.011) 

NA08201 

S2 

40 

0 

.  54  4 

(0 

.090) 

2 

.  959 

(0 

.502) 

0 

.218 

(0 

.008) 

NA08301 

S3 

10 

1 

.  401 

(0 

.047) 

0 

.  235 

(0 

.026) 

0 

.  290 

(0 

.007) 

NA08302 

S3 

11 

0 

.  763 

(0 

.030) 

- 1 

.  618 

(0 

.068) 

0 

.3  51 

(0 

.019) 

NA08303 

S3 

12 

0 

.  865 

(0 

.034  ) 

- 1 

.  368 

(0 

.060) 

0 

.421 

(0 

.016) 

NA0830A 

S3 

13 

1 

.  044 

(0 

.038) 

- 1 

.  270 

(0 

.055) 

0 

.  4 15 

(0 

.017) 

NA08A01 

S3 

14 

0 

.  294 

(0 

.036) 

-1 

.361 

(0 

.166) 

0 

.23  8 

(0 

.019) 

NA08501 

S3 

15 

0 

.  899 

(0 

.039) 

-0 

.  916 

(0 

.045) 

0 

.  251 

(0 

.015) 

NA08502 

S3 

16 

0 

.  756 

(0 

.037) 

0 

.  3  89 

(0 

.027) 

0 

.  150 

(0 

.008) 

^^08601 

S3 

17 

0 

.  602 

(0 

.022) 

-0 

.  628 

(0 

.028) 

0 

.  185 

(0 

.011) 

.<A08701 

S3 

18 

0 

.391 

(0 

.030) 

-0 

.088 

(0 

.020) 

0 

.244 

(0 

.011) 

N408801 

S3 

19 

0 

.418 

(0 

.024) 

0 

.077 

(0 

,017) 

0 

.291 

(0 

.009) 

N408901 

S3 

20 

1 

.  172 

(0 

.059) 

-0 

.422 

(0 

.036) 

0 

.494 

(0 

.011) 

N408902 

S3 

21 

1 

.  118 

(0 

.  040) 

-1 

.646 

(0 

.071) 

0 

.442 

(0 

.023) 

NA08903 

S3 

22 

0 

.809 

(0 

.034) 

-0 

.  160 

(0 

.022) 

0 

.357 

(0 

.010) 

NA0890A 

S3 

23 

0 

.652 

(0 

.030) 

0 

.319 

(0 

.027) 

0 

.435 

(0 

.008) 

NA09001 

S3 

24 

0 

.967 

(0 

.058) 

-0 

.420 

(0 

.039) 

0 

.168 

(0 

.015) 

N409101 

S3 

25 

0 

.646 

(0 

.026) 

-0 

.974 

(0 

.043) 

0 

.213 

(0 

.014) 

N409102 

S3 

26 

0 

.556 

(0 

.026) 

-0 

.178 

(0 

.019) 

0 

.215 

(0 

.010) 

N409103 

S3 

27 

0 

.224 

(0 

.030) 

3 

.708 

(0 

.494) 

0 

.274 

(0 

.006) 

N409201 

S3 

28 

0 

.667 

(0 

.041) 

0 

.048 

(0 

.022) 

0 

.347 

(0 

.009) 

N409301 

S3 

29 

1 

.027 

(0 

.039) 

-0 

.217 

(0 

.021) 

0 

.162 

(0 

.010) 

-608- 

ERIC 


Tabl*  E.3S 
(continued) 


FIELD       BLOCK  ITEM 

N409501  S3  33 
N«09601  S3  3A 
NA09701       S3  35 


A  SE  B 

0.923  (0.027)  1.590 

1.157  (0.158)  0,762 

0.878  (0.041)  1.230 


SE                C  SE 

(0.056)  0.138  (0.005) 

(0,120)  0.282  (0.010) 

(0.066)  0.255  (0,007) 


-609- 
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Tabid  E.36 

1966  IRT  Parameters,  Science  Trend  Items,  Age  17 


FIELD 

BLOCK 

ITEM 

A 

SE 

N400201 

SI 

12 

0. 

A25 

( 0 . 

02A ) 

N404601 

SI 

13 

0. 

A20 

( 0 . 

023 ) 

N410003 

SI 

16 

0. 

386 

060 ) 

N410004 

SI 

17 

0. 

515 

(0 . 

057 ) 

N409901 

SI 

18 

0. 

716 

(Q 

058) 

N408601 

SI 

19 

0. 

602 

(  0 

022) 

N409301 

SI 

20 

1. 

027 

( 0 . 

039 ) 

NA06301 

SI 

21 

0. 

237 

(0 . 

023 ) 

NA06302 

SI 

22 

0. 

A35 

(0  . 

027 ) 

N406303 

SI 

23 

0. 

600 

( 0  . 

030 ) 

N406304 

SI 

2A 

0. 

A60 

(  0 

027 ) 

N410101 

SI 

25 

0. 

713 

(  0 

076) 

NA10102 

SI 

26 

0. 

279 

( 0  . 

036 ) 

NA10103 

SI 

27 

0. 

A29 

( 0  . 

0  A  7 ) 

NA06601 

SI 

28 

0. 

376 

(  0 

021 ) 

KA05001 

SI 

29 

0. 

388 

( 0 . 

021 ) 

NA01201 

SI 

30 

0. 

707 

(0  . 

030 ) 

N405201 

SI 

31 

0. 

37A 

(0  . 

021 ) 

N4 10201 

SI 

32 

0. 

763 

(0  . 

036) 

N406001 

SI 

33 

0. 

926 

(0  . 

039 ) 

NA09501 

SI 

3A 

0. 

923 

(0  . 

027 ) 

NA06101 

SI 

35 

0. 

689 

(0 

029) 

N^»06201 

SI 

37 

1. 

129 

( 0 

033 ) 

NA08101 

SI 

38 

1 

052 

(0  . 

0  A9 ) 

N^»06^»01 

S2 

10 

0 

A86 

(0 . 

031 ) 

NA06A02 

S2 

11 

0. 

62A 

(0 . 

030) 

NA06A03 

S2 

12 

0. 

7  22 

(0 

032) 

NA06A0A 

S2 

13 

0. 

966 

(0 

0A7 ) 

NA06A05 

S2 

lA 

0 

613 

(0 

031) 

NAlOAOl 

S2 

15 

0 

265 

(0 

026) 

NA06801 

S2 

16 

0 

928 

( 0 

033 ) 

NA06802 

S2 

17 

0 

A7A 

(0 

057 ) 

NA06803 

S2 

16 

0 

835 

(0 

031) 

NA0680A 

S2 

19 

0 

726 

(0 

028 ) 

NA06805 

S2 

20 

1 

259 

(0 

059) 

NA06806 

S2 

21 

0 

326 

(0 

023) 

NA10501 

S2 

22 

0 

26A 

(0 

023 ) 

KA10601 

S2 

23 

1 

A57 

(0 

038 ) 

NA 10602 

S2 

2A 

0 

5A3 

(0 

059 ) 

NA 10603 

S2 

25 

1 

1A6 

(0 

0A5) 

NA1060A 

S2 

26 

0 

A56 

( 0 

.  0A9) 

NA06901 

S2 

27 

0 

526 

(0 

.  02A ) 

NA07A01 

S2 

28 

0 

532 

(0 

.  OA  1 ) 

NA07A03 

S2 

30 

0 

.561 

(0 

.  0A9) 

NA07A0A 

S2 

31 

0 

.  A95 

(0 

.  062) 

NA07201 

S2 

32 

0 

.  AlA 

(0 

.  025) 

NA07001 

S2 

33 

0 

.280 

(0 

.  021 ) 

NA10701 

S2 

3A 

0 

.858 

(0 

.  03A ) 

NA07701 

S2 

35 

0 

.  A56 

(0 

.  023) 

NA07301 

S2 

36 

0 

.316 

(0 

.  027 ) 

N407302 

S2 

37 

0 

.  271 

(0 

.  026) 

NA07101 

S2 

38 

0 

.  998 

(0 

.  032) 

NA10801 

S2 

39 

0 

.  5A7 

(0 

.  036) 

NA10901 

S2 

AO 

1 

.21A 

(0 

.  036) 

NA 11001 

S2 

Al 

0 

.  990 

(0 

.  OAl) 

NA08301 

S3 

10 

1 

.  AOl 

(0 

.  0A7) 

HA08302 

S3 

11 

0 

.763 

(0 

.  030) 

HA08303 

S3 

12 

0 

.  865 

(0 

.  03A ) 

NA0830A 

S3 

13 

1 

.  OAA 

(0 

.  038) 

NA05101 

S3 

lA 

0 

.783 

(0 

.029) 

NA08901 

S3 

15 

1 

.  172 

(0 

.059) 

NA08902 

S3 

16 

1 

.  118 

(0 

.040) 

NA08903 

S3 

17 

0 

.809 

(0 

.034) 

N40890A 

S3 

18 

0 

.  652 

(0 

.030) 

NA05A01 

S3 

19 

0 

.769 

(0 

.029) 

NA11301 

S3 

20 

0 

.613 

(0 

.070) 

NA05501 

S3 

21 

0 

.  A97 

(0 

.027) 

NAlllOl 

S3 

22 

0 

.629 

(0 

.030) 

NA11201 

S3 

23 

0 

.  768 

(0 

.033) 

NA08801 

S3 

2A 

0 

.  A18 

(0 

.02A) 

NAllAOl 

S3 

25 

1 

.780 

(0 

.049) 

NA11501 

S3 

26 

1 

.1A5 

(0 

.032) 

NA11502 

S3 

27 

0 

.700 

(0 

.059) 

B  SE  C  SE 


-1. 

954 

(0. 

114  ) 

0. 

229 

(0. 

016) 

-1. 

199 

(0. 

069) 

0. 

270 

(0. 

013) 

-3. 

255 

(0. 

505) 

0. 

413 

(0. 

035) 

-1 . 

290 

(0. 

148) 

0. 

419 

(0. 

025) 

-0. 

678 

(0. 

064) 

0. 

228 

(0. 

024  ) 

-0. 

628 

(0. 

028) 

0. 

185 

(0. 

Oil) 

-0. 

217 

(0. 

021) 

0. 

162 

(0. 

010) 

-3. 

920 

(0. 

386) 

0. 

432 

(0. 

016) 

-0. 

015 

(0. 

021) 

0. 

434 

(0. 

009) 

0. 

360 

(0. 

028) 

0. 

393 

(0. 

008) 

-0. 

043 

(0. 

020) 

0. 

377 

(0. 

009) 

-0  . 

583 

(0. 

074  ) 

0. 

427 

(0. 

022) 

-1. 

015 

(0. 

135) 

0. 

425 

(0. 

019) 

-1. 

569 

(0. 

177) 

0. 

408 

(0. 

025) 

-1. 

155 

(0. 

066) 

0. 

202 

(0. 

012) 

-0  . 

045 

(0  . 

015) 

0. 

205 

(0. 

009) 

0  . 

014 

(0. 

019) 

0 . 

242 

(0. 

009) 

-0. 

205 

(0. 

019) 

0. 

223 

(0. 

010) 

1 . 

638 

(0. 

088) 

0. 

226 

(0. 

Oil) 

2. 

078 

(0  . 

100) 

0  . 

226 

(0. 

005) 

1 . 

590 

(0  . 

056) 

0  . 

138 

(0. 

005) 

2  . 

085 

(0. 

094) 

0. 

185 

(0. 

005) 

2. 

082 

(0. 

077  ) 

0  . 

133 

(0. 

004  ) 

2. 

229 

(0  . 

126) 

0  . 

202 

(0. 

006) 

-0  . 

236 

(0  . 

026) 

0  . 

471 

(0  . 

009) 

-0 

369 

(0  . 

027 ) 

0  . 

339 

(0. 

Oil) 

-1 

552 

(0 

074  ) 

0  . 

411 

(0. 

017  ) 

-0 

481 

(0. 

034  ) 

0  . 

426 

(0. 

Oil) 

-1 

014 

(0 

056) 

0 

387 

(0. 

014  ) 

-0 

362 

(0 

041) 

0 

270 

(0 

015) 

-1 

612 

(0 

065) 

0 

357 

(0 

020) 

3 

024 

(0 

367) 

0 

548 

(0 

005) 

-1 

080 

(0 

046) 

0 

315 

(0 

014  ) 

"1 

292 

(0 

055) 

0 

300 

(0 

015) 

1 

575 

(0 

106) 

0 

611 

(0 

005) 

-0 

114 

(0 

020 ) 

0 

349 

(0 

010) 

-0 

635 

(0 

059) 

0 

190 

(0 

016) 

1 

680 

(0 

071) 

0 

164 

(0 

008) 

-2 

006 

(0 

224) 

0 

412 

(0 

032) 

1 

023 

(0 

061) 

0 

361 

(0 

Oil) 

-1 

.955 

(0 

214) 

0 

407 

(0 

029) 

-0 

.296 

(0 

021) 

0 

201 

(0 

010) 

-0 

.249 

(0 

.036) 

0 

391 

(0 

017) 

-0 

.282 

(0 

.041) 

0 

.416 

(0 

018) 

-1 

.051 

(0 

.  136) 

0 

.418 

(0 

023) 

0 

.  556 

(0 

.037) 

0 

.229 

(0 

.008) 

-0 

.069 

(0 

.017) 

0 

.206 

(0 

.010) 

1 

.  143 

(0 

.058 ) 

0 

.218 

(0 

.  012) 

0 

.  642 

(0 

.037  ) 

0 

.  164 

(0 

.  008) 

1 

.  903 

(0 

.  165) 

0 

.  259 

(0 

.  008 ) 

1 

.448 

(0 

.  143) 

0 

.277 

(0 

.  009) 

1 

.  561 

(0 

.062) 

0 

.  140 

(0 

.  006) 

2 

.  268 

(0 

.155) 

0 

.  227 

(0 

.  010 ) 

1 

.  530 

(0 

.064  ) 

0 

.  122 

(0 

.  009) 

2 

.  277 

(0 

.  113 ) 

0 

.  142 

(0 

.  007  ) 

0 

.235 

(0 

.  026) 

0 

.  290 

(0 

.  007 ) 

-1 

.  618 

(0 

.068) 

0 

.351 

(0 

.019) 

-1 

.368 

(0 

.060) 

0 

.  421 

(0 

.016 ) 

-1 

.  270 

(0 

.055) 

0 

.415 

(0 

.  017  ) 

0 

.  350 

( 0 

.  022) 

0 

.  216 

(0 

.  007  ) 

-0 

.422 

(0 

.036) 

0 

.494 

(0 

.011) 

-1 

.646 

(0 

.071) 

0 

.442 

(0 

.023) 

-0 

.  160 

(0 

.022) 

0 

.357 

(0 

.010) 

0 

.319 

(0 

.027) 

0 

.435 

(0 

.008) 

0 

.821 

(0 

.038) 

0 

.  166 

(0 

.007  ) 

4 

.  181 

(0 

.493) 

0 

.  142 

(0 

.007) 

0 

.813 

(0 

.048) 

0 

.208 

(0 

.008) 

0 

.801 

(0 

.  049) 

0 

.  190 

(0 

.015) 

0 

.822 

(0 

.047  ) 

0 

.214 

(0 

.014) 

0 

.077 

(0 

.017) 

0 

.291 

(0 

.009) 

1 

.201 

(0 

.066) 

0 

.216 

(0 

.011) 

1 

.800 

(0 

.070) 

0 

.179 

(0 

.009) 

-0 

.604 

(0 

.063) 

0 

.255 

(0 

.026) 

ERIC 


-610- 


Table  E.36 
(continued) 


FIELD 

BLOCK 

ITEM 

A 

SE 

B 

SE 

c 

SE 

NA11601 

S3 

26 

1 

29A 

(0 

.OAO) 

1 

652 

(0 

.076) 

0 

225 

(0 

.010) 

NA11701 

S3 

29 

0 

885 

(0 

.03A) 

1 

850 

(0 

.085) 

0 

19A 

(0 

.010) 

NA11801 

S3 

30 

1 

782 

(0 

.0A8) 

1 

1A6 

(0 

.062) 

0 

192 

(0 

.011) 

NA11901 

S3 

31 

1 

078 

(0 

.035) 

1 

859 

(0 

.079) 

0 

202 

(0 

.009) 

NA12001 

S3 

32 

0 

7A7 

(0 

.0A6) 

2 

5A3 

(0 

.175) 

0 

303 

(0 

.010) 

ERIC 
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APPENDIX  F 
U.S.  History  and  Literature  Items 


Table  F.l 

History  and  Literature  Derived  Variables 


YRSHIST  (Years  of  History  and  Related  Courses)  . 

Items  H800201  to  H800205  asked  students  to  indicate  the  number  of  years 
chat  they  had  studied  a  particular  history  or  history- related  course.  Each 
of  the  five  items  was  recoded  as  follows: 

Studied  1  school  year  -  1.0 

Studied  1/2  school  year  =  .5 

Studied  less  that  1/2  school  year  -«  .25 
Have  not  studied  ^  0 

The  sum  of  the  five  recoded  variables  was  then  assigned  the  codes: 

0  to  1  years  «  1 

More  than  1  to  2  years  *  2 

More  than  2  to  3  years  =  3 

More  thaa  3  years  =  4 


NHIST  (Number  of  Topics  Studied) 

Items  H800301  to  H80O306  asked  students  if  they  had  taken  various  topics 
since  9th  grade.     The  sum  of  the  number  of  topics  taken  was  computed  and 
recoded  as  follows : 


0-2  topics  studied  =  1 

3  topics  studied  «  2 

4  topics  studied  «  3 

5  topics  studied  =  4 

6  topics  studied  *  5 


NHIST2  (Number  of  Topics  Studied) 

This  variable  was  created  from  the  same  as  items  used  for  NHIST,  but 
were  recoded  as : 

0-2  topics  studied  -=  1 
3-4  topics  studied  «  2 
5  -  6  topics  studied    «  3 


Table  F.l  (continued) 


HISLEN  (How  Long  Since  You  Took  a  U.S.  History  Course) 

Item  H800101  was  recoded  as: 

Taking  one  now  =  1 

1-2,  3-4  years  ago        -  2     (1-4  years  ago) 

Haven't  taken  one  «  3 

USHISl  to  USHIS6  (How  Often  Do  Things  in  U.S.  History  Course) 

Responses  to  items  H800401  to  H800406  were  collapsed  as  follows: 

Once  a  week,  once  a  month, 

several  times  a  year  «  1  (ever) 

Hardly  ever  or  never  -=  2  (never) 

HISl  to  HIS7  (How  Often  Do  Things  in  History  Course) 

Responses  to  items  H800501  to  H8G0507  were  collapsed  as  follows: 

Every  day,  2-3  times  a  week, 

OTice  a  week,  <  once  a  week      «  1  (ever) 
Never  2  (never) 

NWKSCH  (Number  of  Works  Read  for  School) 

Items  L800601  to  L800607  asked  students  to  indicate  the  number  of  works 
they  had  read  for  school  during  the  first  half  of  the  school  year.  First, 
responses  to  each  of  the  seven  items  were  recoded  as  follows: 

None  0 
1-2  -  1.5 

3-4  -  3.5 

5-6  -  5.5 

More  than  6-7.5 

The  sura  of  the  seven  recoded  variables  was  then  assigned  the  values: 

5  or  less  -  1 

6  to  10  -  2 
11  to  15  -  3 
16  to  20  -  4 
21  to  25  -  5 
Greater  than  25      -  6 
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Table  F.l  (continued) 


NWKOWN  (Ntimber  of  Works  Read  on  Your  Own) 

Items  L800701  to  L800705  asked  students  to  indicate  the  nximber  of  works 
they  had  read  on  their  own  during  the  first  half  of  the  school  year.  The 
five  values  were  recoded  in  the  same  way  as  were  the  component  variables  used 
in  NWKSCH,    The  sum  of  these  five  recoded  variables  was  then  assigned  the 
values : 

5  or  less  -  1 

6  to  10  -  2 
11  to  15  -  3 
Greater  than  15  «  4 


FRQPRAL  (Frequency  of  Classroom  Practices) 

Items  L801201  to  L801209  asked  students  to  indicate  if  their  English 
teacher  practiced  various  activities.  The  sum  of  the  nximber  of  practices 
conducted  is  assigned  the  values: 

Low  (0-5  practices)  -  1 

Medixim  (6  or  7  practices)  «  2 
High  (8  or  9  practices)      -  3 

NBOOKS  (Ntimber  of  Books  Read  on  Own  or  for  School) 

Items  L801401  to  L801410  asked  students  to  indicate  whether  they  had 
read  specific  books  either  for  school,  on  their  own,  or  not  at  all.     The  sum 
of  the  number  of  books  that  the  student  had  read  on  his  or  her  own  or  read 
for  school  was  assigned  the  values: 

0  -  1  books  -  1 


2-3  books 

4  books 

5  books 

6  or  more  books 


2 
3 
4 
5 
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Table  F.l  (continued) 


NBKSCH  (N\amber  of  Books  Read  for  School) 

Items  L801401  to  L801410  asked  students  to  indicate  whether  they  had 
read  specific  books  either  for  school,  on  their  own,  or  not  at  all.     The  stun 
of  the  number  of  books  that  the  student  had  read  for  school  was  assigned  the 
values : 

0  books  1 

1  book  -  2 

2  books  -=  3 

3  books  -  4 

4  or  more  books  5 


PERLIT  (In  English  Class,  Percentage  of  Time  Spent  on  Literature) 

Item  L800101  was  recoded  as  follows: 

<  25%,  about  25%         -    1      (25%  or  less) 

About  50%  -    2  (50%) 

About  75%,  >  75%         -    3      (75%  or  more) 


WRITLIT  (Does  Your  English  teacher:  Plot  and  Analyses) 

Responses  to  item  L801207  (Does  your  English  teacher  ask  you  to  write 
summaries  [plot])  and  item  L801208  (Does  your  English  teacher  ask  you  to 
write  analyses)  were  combined  as  follows: 

Plot  -  no  ,  analysis  -  no  -  1 

Plot  -  yes,  analysis  -  no  -  2 

Plot  -  no  ,  analysis  -  yes  «  3 

Plot  -  yes,  analysis  -  no  -  4 


ERLC 
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Table  F,2 


NAEP  ID  Numbers  for  Items  Used  In 
Mean  Percents  Correct  In  Literature  and  U.S.  History 


Women's  History 

H005801 
H002501 
H009401 
H005301 
H002101 
H006901 
H007301 
H006401 

Black  History 

H002601 
H002701 
H003101 
H003401 
H004301 
H005801 
H006001 
H006101 
H007401 
H009201 
H009601 
H010601 
H010801 
H001601 


Other  Social  Trends 
&  Movements 

H000701 
H001301 
H001401 
H001501 
H002301 
H003201 
H005701 
H007201 
H007301 
H008601 
H009301 
H009501 
HOIOIOI 
H009901 
H005501 
H007801 
HOOlOOl 
H004001 
H007901 


People 

H000801 
HOOllOl 
H001601 
H002001 
H002901 
H003001 
H004301 
H004601 
H004701 
H005801 
H006201 
H006401 
H006601 
H006901 
H007501 
H007601 
H008501 
H008701 
H009401 
H009701 
H010801 
H010901 
HOlllOl 

Black  Leaders 


Documents 

H000201 

H000901 

H001701 

H001801  ■ 

H002801 

H003501 

H006301 

H006501 

H006801 

H008101 

H009101 

H010201 

H011201 


H005801 
H004301 
H010801 
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Table  F.2  (continued) 


Slavery  and  Civil 
Rights 

H006101 
H001601 
H006001 
H006301 
H009201 
H003101 
H003401 
H010601 
H007401 
H009601 
H002701 

Civil  War 
&  Reconstruction 

HOllOOl 
HOOOlOl 
H002601 
H005007 
H002403 
H005101 
H006701 

Hispanic  History 

HOOlOOl 
H007103 

Exploration  &  Early 
Colonization 

H005010 
H005901 
H000401 
H005201 
H008201 
H008401 
H008801 
H002201 
H004201 


Revolutionary  War  Era 

H010201 
H000901 
H003001 
H006801 
H009101 
H005103 
H005401 
H006501 
H007501 
H000201 
H011401 

Constitution  &  the 
New  Government 

H000501 
HOOllOl 
H002701 
H003501 
H005006 
HOlOOOl 
H011201 
H002402 

Territorial  Expansion 
&  Foreign  Policy 

H001701 
H010701 
H000601 
H001901 
H003301 
H003801 
H005601 
H008001 
H008101 
H007001 
H003701 
H003901 
H004101 
H007101 
H007102 
H007103 


World  War  II 

H010301 
H009801 
H000301 
H007701 
H008304 
H008303 
H005004 
H008302 
H008305 
H003601 
H001201 
H001204 
H001205 
H001202 
H001203 
H004701 
H002901 
H010901 

Maps 

HOOOlOl 
H001201 
H001202 
H001203 
H001204 
H001205 
H004501 
H004502 
H007101 
H007102 
H007103 
HOlOOOl 
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Chronology 

HOOlOOl 
H002101 
H002401 
H002402 
H002403 
H002404 
H002405 
H002406 
H002407 
H002408 
H004001 
H005101 
H005102 
H005103 
H005301 
H007901 
H008301 
H008302 
H008303 
H008304 
H008305 
H005004 
H005005 
H005006 
H005007 
H005008 
H005009 
H005010 
H010401 
H010501 

Black  Literature 

L002501 
L005201 
L007601 
LOIOIOI 
L010901 


Table  F.2  (continued) 


Biblical  Characters 
&  Stories 

L000401 
L001201 
L002101 
L002701 
L003401 
L004201 
L005101 
L005701 
L006501 
L008201 
L011201 
L003801 
L009501 
L007301 
L010301 

Shakespeare 

L009701 
L000601 
L000201 
L004501 
L008401 
L003601 
L006701 

Classical  Myths. 
Legends  &  Epics 

L000701 
LOO 1401 
L001501 
L002901 
L003701 
L005401 
L005901 
L006101 
L006801 
L007701 
L008501 
L009801 
LOlllOl 
L011501 
L011801 
L005801 
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